AI2 drops Molmo 2 to conquer video understanding

AI2's Molmo 2 is an open-source multimodal model family built for video understanding, pointing, and object tracking. It natively accepts single images, multi-image inputs, and video clips of varying lengths, pinpointing visual events with timestamps and spatial coordinates rather than just spitting out descriptive text.

The release comes in three sizes. Molmo 2 (8B) is based on Qwen 3 and serves as the flagship for video grounding and QA. Molmo 2 (4B) is also Qwen 3-based and tuned for efficiency. For teams demanding total transparency across the entire model pipeline, Molmo 2-O (7B) is built on Olmo, giving researchers end-to-end control over the vision encoder, connector, and language model.

What it does

Where most multimodal models only provide high-level summaries, Molmo 2 grounds answers directly in space and time. Ask it how many times a robot grasps a block, and it returns coordinates and timestamps for every single grasp event instead of just a raw number.

It handles multi-object tracking with persistent IDs across occlusions and re-entries, dense video captioning, subtitle-aware QA, and visual artifact detection in generative video. On evals, Molmo 2 (8B) beats open-weight tracking baselines and outperforms Gemini 3 Pro by a wide margin. It also tops all open-weight models on an 11-benchmark image average, places just behind GPT-5 and ahead of Gemini 2.5 Pro, and actually outperforms the original 72B Molmo on core image pointing benchmarks.

Under the hood and open release

AI2 achieved this while training on just 9.19 million videos—less than one-eighth of the 72.5 million videos Meta used for PerceptionLM. The secret lies in nine new, heavily curated datasets centered on dense captioning, where human narrators provided rich descriptions supplemented by frame-level details.

Architecturally, Molmo 2 uses a vision transformer paired with a lightweight connector that interleaves visual tokens with timestamps and text. Visual tokens attend bi-directionally across frames, allowing joint reasoning over space and time. During inference, it handles up to 128 frames at ≤2 fps and supports a SlowFast sampling strategy to process long clips without token blowup.

AI2 released the complete stack: weights, data prep scripts, pretraining and SFT code, evaluation tools, and inference support for both transformers and vLLM.

Why it matters

If you are building robotics workflows, video search engines, or automated inspection tools, Molmo 2 is worth testing immediately. Packaging state-of-the-art spatio-temporal tracking into an 8B model means you can run serious video intelligence on local developer hardware without paying proprietary API fees.

In our view, AI2 proved that careful data curation and explicit grounding objectives completely trump raw video volume. While ultra-long video reasoning still favors massive proprietary models, Molmo 2 sets a formidable open-source benchmark for local video grounding and verifiable vision workflows.