Alibaba's new AI agent learns to read maps like a human

Anyone who has spent time on GeoGuessr knows that identifying a specific utility pole or vegetation type is only half the battle. The real trick is panning around Google Maps to verify your hunch against reality.

AI models haven't been playing the game like that. Despite massive world knowledge and reasoning chains, most vision-language models still treat geolocalization like a memory test or a generic text-search problem. On January 8, Alibaba researchers introduced Thinking with Map, an agentic framework designed to teach multimodal models how to actually use a map.

Putting the agent inside the map

Instead of asking the model to guess coordinates straight from an image, the researchers formulated an agent-in-the-map loop that mimics human investigation.

The system relies on a two-stage optimization scheme. First, agentic reinforcement learning teaches the model efficient sampling and navigation strategies when interacting with a map. Second, parallel test-time scaling allows the agent to spin up multiple candidate reasoning paths simultaneously before locking in a final location.

To see how it handles messy real-world photos, the team built MAPBench, an evaluation benchmark made entirely of in-the-wild images.

Crushing frontier models on precision

The dual setup paid off dramatically on benchmark numbers.

On MAPBench, Thinking with Map surpassed existing open- and closed-source systems across most evaluation metrics. Most notably, its 500-meter accuracy hit 22.1%.

By comparison, Gemini-3-Pro operating in its grounded mode with Google Search and Maps managed just 8.0% on the exact same benchmark. That gives Alibaba's agent a 2.8x performance advantage when pinpointing a location down to a specific neighborhood.

Why it matters

For real-world investigative tasks, fine-grained geolocalization is a massive deal. Open-source intelligence researchers, digital forensics experts, and fact-checkers spend hours cross-referencing street furniture and satellite imagery; automating even part of that verification workflow saves serious legwork.

For the AI industry, this paper is another proof point that test-time compute scaling is moving beyond code and math into complex tool manipulation. Arming an agent with interactive domain tools and letting it parallel-search paths beats relying on static pre-trained weights every time.

We think Alibaba's approach is both intuitive and strategically sharp, likely leveraging internal mapping DNA to carve out a distinct agentic edge. Still, a 22.1% accuracy at the 500-meter mark means roughly four out of five difficult wild shots still fail to land on the exact block. Running parallel exploration loops during inference also introduces real compute overhead, meaning cost and latency will be the big hurdles before this hits everyday applications.