The AI models that hallucinate the least

Conventional wisdom in AI says that bigger models with more parameters are smarter, and therefore less likely to lie to your face.

Turns out, that assumption doesn't hold up. According to fresh testing data from Vectara visualized by Visual Capitalist, the models least prone to hallucinations are actually nimble, lightweight contenders.

In the ranking updated through December 11, 2024, China's Zhipu AI with its open GLM-4-9B-Chat and Google's Gemini-2.0-Flash-Exp tied for first place with an ultra-low hallucination rate of 1.3%. OpenAI's specialized reasoning shrink, o1-mini, took third place right behind them at 1.4%.

Small models stick to the facts

Vectara defines hallucination rate as how often an LLM spits out false or unsupported statements. IBM famously describes this as a model detecting phantom patterns that do not exist, leading to confident nonsense.

To measure this, Vectara had each model summarize 1,000 short documents, then deployed an evaluation model to catch factually inconsistent outputs.

The biggest takeaway is how well smaller footprints performed. Intel's Neural-Chat-7B clocked a 2.6% hallucination rate, Microsoft's Phi-3.5-MoE-instruct hit 2.5%, and Alibaba's Qwen2.5-7B-Instruct landed at 2.8%.

Vectara noted that smaller architectures can achieve hallucination rates comparable to, or even better than, massive frontier models.

While brute-force large models still excel at raw power, they suffer from high inference latency, massive costs, and complex routing. In fact, earlier studies pointed out that leaner models like Mistral 8x7B successfully cut down on generated hallucinations by keeping tasks bounded.

How OpenAI and Google stack up

Among the top tier, Google's experimental Gemini 2.0 Flash slightly edged out OpenAI's flagship GPT-4 by 0.2 percentage points.

Yet OpenAI showed remarkable consistency across its entire portfolio. GPT-4o landed at 1.5%, GPT-4o-mini and GPT-4-Turbo tied at 1.7%, standard GPT-4 sat at 1.8%, and even legacy GPT-3.5-Turbo managed 1.9%.

That narrow spread between 1.5% and 1.8% demonstrates that OpenAI has mastered factual alignment techniques across diverse tiers of the same architecture.

DeepSeek-V2.5 also made a strong showing at 2.4%, outperforming Microsoft's Orca-2-13b and underscoring how competitive open-weight alternatives have become.

Why it matters

For builders and enterprise teams building RAG pipelines, contract summarizers, or financial bots, this data is an immediate green light to downsize.

You simply do not need to burn API credits on giant models if your primary goal is factual fidelity on document processing. A model like GLM-4-9B or GPT-4o-mini delivers fewer hallucinations at a fraction of the latency and dollar cost.

For the broader industry, it challenges the brute-force scaling myth. When deployment enters regulated sectors like healthcare, law, and banking, a model that strictly refuses to invent facts is far more valuable than a sprawling model that occasionally daydreams.

That said, we think it is crucial to remember the methodology here. This benchmark tested 1,000 short document summaries—essentially an open-book test where smaller models naturally excel at tight context retention. Whether these lightweight models hold up just as reliably in open-ended zero-shot queries without source text remains an open question.