Google's Gemini 3 Pro tackles spatial reasoning

Most multimodal models are fine at telling you there is a dog in a photo, but they fall apart when handed a messy 60-page PDF report with nested charts or a fast-moving sports video.

Google took the wraps off Gemini 3 Pro's upgraded vision capabilities. Google frames this update as a generational leap from basic visual recognition to genuine visual and spatial reasoning. The model scored new highs on benchmarks like MMMU Pro and Video MMMU, focusing on four primary areas: messy documents, spatial precision, screen UI, and high-frame-rate video.

De-rendering messy pages and pixel-precise pointing

Real-world documents are notoriously tricky for AI because of irregular layouts, interleaved graphics, and handwriting. Gemini 3 Pro introduces what Google calls "derendering"—essentially reverse-engineering visual pages directly into structured HTML, Markdown, or LaTeX. The model can parse an 18th-century merchant ledger into a clean table or convert scratched math formulas into clean code.

On the CharXiv reasoning benchmark, Gemini 3 Pro hit 80.5%, outperforming the human baseline. Google demonstrated this by having the model analyze a dense 62-page U.S. Census Bureau report, tracking changes in the Gini index and identifying the economic drivers behind discrepancies across tables.

Spatial awareness also received a major upgrade. Gemini 3 Pro can now output pixel-precise 2D coordinates. This allows it to trace human pose trajectories or guide robotics with open-vocabulary commands, like generating a step-by-step sorting plan for a cluttered tabletop. In AR settings, it can literally point out which screw to turn based on an instruction manual.

That same spatial accuracy powers desktop and mobile screen understanding. The model accurately detects UI components and registers clicks, laying a stronger foundation for computer-use agents handling QA testing or repetitive workplace workflows.

10 FPS video analysis and token cost controls

Video understanding usually suffers because models sample at very low frame rates to save computation. Gemini 3 Pro pushes sampling past 1 FPS up to 10 FPS—ten times the standard rate. This allows it to capture rapid, micro-movements, demonstrated in an analysis of golf swing mechanics and weight shifts.

Combined with an upgraded "thinking" mode, the model traces cause-and-effect timelines across footage rather than merely tagging objects. It can also translate long video walkthroughs directly into working code.

For builders watching their cloud budgets, Google introduced a new media_resolution parameter alongside native aspect ratio preservation. Developers can toggle between high resolution for dense OCR and low resolution for general scene understanding, giving direct control over visual token consumption and latency.

In domain-specific tests, the model set state-of-the-art results across medical benchmarks, including MedXpertQA-MM, radiology-focused VQA-RAD, and microscopy research benchmark MicroVQA.

Why it matters

We think Google made a very calculated move here by targeting high-friction enterprise bottlenecks instead of chasing novelty demos. Complex financial charts, UI automation, and rapid video inspection are where multimodal AI can actually justify its compute costs today.

For developers, the media_resolution setting is arguably the most practical addition. High-fidelity visual tokens get expensive fast; letting developers downsample non-critical frames provides a realistic pathway to running video agents without blowing through margins. Furthermore, being able to de-render documents into code eliminates fragile pipelines stitched together with multiple niche OCR tools.

Competitive pressure is mounting directly against Anthropic's Computer Use and OpenAI's visual offerings. Google's native advantage has always been its rich video infrastructure, and bringing 10 FPS reasoning to production models pushes the frontier on physical-world and dynamic visual comprehension.

The real test will be latency and operational expense. Catching every frame of a fast-moving swing or parsing a 60-page PDF with multi-step reasoning requires massive compute. While the capabilities look stellar on paper, developers will need to carefully balance resolution settings before deploying these agents at scale.