Gemini's 1M context cuts Cody's code hallucinations in half

If you have ever tried pasting a massive, tangled codebase into an AI coding assistant, you know the frustration. Conventional context windows force tools to guess which snippets you need, and the AI usually ends up hallucinating APIs that do not exist.

Google and Sourcegraph just shared the results of plugging a full 1M token context window from Gemini 1.5 Flash into the Cody coding assistant, running it against tough technical questions on complex codebases.

Halving hallucinations, with a speed catch

Sourcegraph compared the 1M token setup directly against Cody's current production pipeline. Across their core benchmarks—Essential Recall, Essential Concision, and Helpfulness—the ultra-long context window delivered across-the-board wins.

The headline metric is the hallucination rate. When dealing with enterprise repositories like those at Palo Alto Networks and Leidos, bad answers can break systems. Expanding the window dropped the hallucination rate from 18.97% down to 10.48%.

There was a massive catch: latency. Time to first token scales linearly with context length, meaning developers originally had to stare at a blank screen for 30 to 40 seconds before the AI even began typing.

Sourcegraph tackled this by building a prefetching mechanism and a layered context architecture that caches model execution states. Paired with Gemini 1.5 Flash and Pro, they managed to slash that initial delay down to around 5 seconds for 1MB of context.

Why it matters

For a long time, the dominant playbook for code assistants has been aggressive RAG, retrieving tiny slices of code and hoping the embedding search picked the right files. Sourcegraph's benchmark proves that brute-force context windows can solve the structural dependency blind spots that standard retrieval consistently misses.

Dropping code hallucinations almost in half is a legitimate breakthrough for teams navigating millions of lines of legacy spaghetti code. It shifts the AI from a toy autocomplete tool into an actual systems analyst.

Still, we think developers should temper their immediate expectations for everyday workflows. A five-second wait time is a heroic engineering achievement compared to 40 seconds, but in a real-time coding session, five seconds is still an eternity when you are in the flow.

Between execution caching complexity and the raw token bills of running million-token queries, this approach looks primed for deep debugging and PR reviews rather than instant autocomplete, at least until inference costs fall further.