Baidu drops open-source ERNIE multimodal model that "thinks with images"

ERNIE-4.5-VL-28B-A3B-Thinking is an open-source multimodal reasoning model from Baidu designed to handle complex charts, document parsing, visual STEM problem-solving, and video analysis.

Instead of treating images as static snapshots, it introduces a dynamic mechanism that inspects visual details much like a human would.

What it can do

Most vision-language models digest images at a rigid, fixed resolution. When faced with dense technical schematics or crowded charts, they often hallucinate small text.

ERNIE-Thinking implements what Baidu calls "Thinking with Images." It dynamically zooms in and out of specific image regions to catch tiny details, like subtle manufacturing defects or obscure graph labels.

When paired with external tools, the system can trigger reverse image search to look up unfamiliar visual data beyond its pre-training boundaries.

It also supports industrial-grade visual grounding, locating specific objects within images with precise coordinates for robotics and warehouse automation.

For video tasks, Baidu claims the model tracks temporal shifts and pinpoints specific events across different video segments.

28B parameters with only 3B active

The model's standout engineering feat is efficiency: while it carries 28 billion total parameters, its routing architecture activates just 3 billion parameters at runtime.

This keeps memory and compute footprints close to a lightweight model while retaining the reasoning breadth of a 28B system.

Baidu says it achieved this through an extensive mid-training phase with rich multimodal reasoning corpora and large-scale reinforcement learning.

On document and chart benchmarks, Baidu claims the model edges past Gemini 2.5 Pro and GPT-5-High, though independent verification from the broader community is still pending.

How to get it

The weights are available directly on Hugging Face under a fully permissive Apache 2.0 license.

Commercial use is unrestricted, making it an attractive drop-in option for teams building proprietary visual pipelines.

Because only 3B parameters light up during inference, you can run it on far leaner hardware than typical frontier models require.

Why it matters

If you deal with automated document processing, chart analysis, or edge-based visual inspection, this is definitely worth a test run.

Baidu made a smart tradeoff here: dynamic zooming solves the classic resolution bottleneck without forcing you to burn massive compute on full-sized image tiles.

Packing that into an Apache 2.0 release gives it a massive edge over restrictive proprietary APIs. Just take the benchmark claims with a grain of salt until third-party evals roll in.