Google DeepMind launches MKA benchmark for visual agents
Google DeepMind and Artificial Analysis just launched MKA, a new benchmark evaluating how AI agents handle visually complex tasks like cross-referencing blueprints and reading charts. Gemini 4 Carbon took second place, while keeping average per-task costs six times lower than the top model.
Most agent benchmarks still test code generation or plain text tasks. But real professional work rarely lives in a clean text terminal.
Testing agents on blueprints and charts
Google DeepMind teamed up with Artificial Analysis to launch MKA (Multimodal Knowledge Agent), a new benchmark built specifically to measure how AI agents tackle visually complex jobs.
Everyday professional workflows frequently involve cross-referencing blueprints or deciphering scientific charts, areas existing evaluations largely ignore.
In the initial test, Google's Gemini 4 Carbon landed in second place. While it didn't take the absolute top spot, its average cost per task came in six times lower than the number-one model.
Why it matters
For developers building visual agents for real-world industries, raw accuracy is only half the battle. If a model lands right behind the leader while cutting operational bills by 6x, it becomes an easy sell for production workflows.
We think Google's focus on unit economics here is smart. As frontier multimodal intelligence matures, cost efficiency will ultimately decide which agents actually get deployed at scale.
Sources (1)
- @GoogleDeepMind(opens in a new tab)Originalx.com