Confluence Labs hits 97.9% on ARC-AGI-2 for under $12

Confluence Labs just open-sourced an ARC-AGI-2 solver that scores 97.9% on the benchmark's public evaluation, spending an average of just $11.77 per task.

The tool uses LLM-guided program synthesis to tackle novel, visual reasoning problems by having off-the-shelf models write, test, and refine code until they uncover the underlying transformation.

How the solver works

ARC-AGI-2 measures how well an AI can solve unfamiliar logic puzzles from only a handful of examples. A year ago, even frontier foundation models hovered in the single digits on this benchmark.

Confluence Labs bypassed brute-force pretraining entirely. Instead of teaching a model intuition, they leaned into what LLMs already do best: writing executable code.

The framework structures each ARC puzzle into a representation that looks like standard training data for coding models. It then directs the model to synthesize programs that mathematically describe the puzzle's transformation.

Critically, the solver supports long-horizon exploration. Rather than guessing once and giving up, the model builds on previous failed attempts, using concrete feedback and exact evaluation criteria to iteratively debug its solution until all demonstration examples pass.

Where to find it

The full solver is publicly available under the confluence-labs/arc-agi-2 repository on GitHub, with instructions for reproducing the benchmark score.

The Y Combinator-backed lab, founded by Brent Burdick, is now looking to adapt this hybrid reasoning engine for real-world domains where gathering data requires costly physical experiments, such as hardware engineering, materials science, and drug discovery.

Why it matters

If you work on reasoning chains, programmatic search, or agentic workflows, this repo is an essential teardown. It shows that scaling test-time compute through structured code synthesis can punch far above raw parameter size on abstraction tasks.

More importantly, with ARC-AGI-2 essentially saturated at under $12 a puzzle, the benchmark era for abstract grid reasoning is nearing its finish line. The real test is whether this exact hypothesis-generation loop can hold up when applied to messy, noisy physical sciences.