Ai2's open web agent browses by sight, not code

MolmoWeb is an open-weight visual web agent from Ai2 that navigates the internet like a human: it looks strictly at browser screenshots, completely ignoring underlying HTML code or accessibility trees.

Developers building web agents usually face a frustrating trade-off between closed, opaque APIs like OpenAI Operator and open frameworks that lack an underlying trained model. MolmoWeb delivers a real alternative as a fully trained, open-weight vision model designed specifically for taking action in browsers.

What it does

MolmoWeb comes in two compact sizes: 4B and 8B parameters. Because it only relies on visual input, it is completely browser-agnostic and can run against local Chrome, Safari, or hosted cloud browser environments.

At each step, the model takes in the user's task prompt, the current screenshot, a log of prior actions, and the page URL and title. It outputs a natural-language thought explaining its plan, followed by an executable action such as clicking coordinates, typing text, scrolling, navigating, or switching tabs.

Ai2 reports that MolmoWeb outperforms other open-weight alternatives across four live-website benchmarks, including WebVoyager, Online-Mind2Web, DeepShop, and WebTailBench. It even beats older GPT-4o setups that relied on accessibility trees combined with screenshots.

How it was built

The real breakthrough alongside the weights is MolmoWebMix, what Ai2 describes as the largest publicly released collection of human web-task execution ever assembled.

It consists of three pillars. The core is 30,000 human task trajectories covering more than 590,000 subtask demonstrations recorded across 1,100 websites using a custom Chrome extension.

To scale further, Ai2 generated synthetic browsing paths using text-only accessibility agents, without relying on proprietary systems like Anthropic's computer use API or OpenAI Operator. Finally, it includes 2.2 million screenshot question-answer pairs across nearly 400 websites to teach the model visual grounding and UI reasoning.

Limitations and availability

MolmoWeb is not without rough edges. The model still occasionally misreads text directly from screenshots, drag-and-drop interactions remain brittle, and performance dips on ambiguous prompts.

It was also deliberately not trained on flows requiring authentication or financial transactions. A hosted demo is available powered by Browserbase, alongside downloadable weights.

Why it matters

If you want to automate browser tasks without paying per-token API taxes or exposing internal enterprise workflows to third-party providers, MolmoWeb is the most complete open alternative available today. An 8B footprint means teams can actually run and fine-tune it on their own hardware.

More importantly, open-sourcing the 30,000 human trajectories alongside the weights breaks open the black box of web agents. Ai2 did not just drop a model; it gave the community the exact recipe needed to audit, adapt, and push visual automation forward.