OmniCapBench
Tencent Hunyuan's NeurIPS 2026 (Evaluations & Datasets track) open benchmark: a deep-structured evaluation framework for fine-grained audio-visual video captioning, Apache-2.0.
Why it matters
Evaluates captioning at four structural levels — shots, subshots, audio events, and dense references — with an open annotation pipeline and scoring toolkit, plus a live leaderboard across nine omnimodal MLLMs.
What you could build with it
A video-model team could run their model against OmniCapBench's public leaderboard to get a fine-grained diagnosis of exactly where captioning breaks down — missed shots, subshots, or audio events — turning vague qualitative evals into concrete, fixable failure modes.
Does it hold up?
Too early to judge — repo created Oct 12 with 5 stars; benchmark data on Hugging Face (OmniCapBench/OmniCapBench, 786 densely annotated videos, 5,818 references, 6,537 audio events); no independent replication reports found yet. Paper: arXiv 2610.12458.
Learn more
First spotted on github: source.
More AI models
OpenAI publishes 722 AI-generated math manuscripts from an unreleased modelOpenAI released 722 mathematical manuscripts in 372 result families to GitHub under Apache-2.0, produced by an…models · JEV 0.74StepFun Step-5-Preview — open BF16 weights releasedStepFun published the full BF16 weights for its flagship Step-5-Preview agentic model (600B-parameter sparse MoE, 27B…models · JEV 0.73ElevenLabs Eleven v4 + v4 Turbo: new emotive TTS architecture with a 100ms real-time variantElevenLabs released Eleven v4 and Eleven v4 Turbo on September 28: a new text-to-speech architecture with inline…models · JEV 0.72Mistral Large 4 (Le Chonk): 1.05T-param open-weight MoEMistral's new flagship: a 1.05T-parameter mixture-of-experts multimodal model with 49B active params and 1M-token…models · JEV 0.72VoiceStudio: open-source, fully-local ElevenLabs alternative with voice cloning, dubbing and dictation in 646 languagesVoiceStudio is an open-source, fully-local voice AI studio: voice cloning from a 3-second sample, voice design, video…models · JEV 0.71JetBrains Mellum 2.1: 12B MoE coding model trained for agentsJetBrains released Mellum 2.1, a 12B MoE model (2.5B active parameters, 128K context) under Apache 2.0, trained mainly…models · JEV 0.69
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.