Vectara's benchmark shows Zhipu AI's GLM-4-9B-Chat and Google's Gemini-2.0-Flash-Exp tied for the lowest hallucination rate at just 1.3%, followed by OpenAI's o1-mini at 1.4%. The numbers prove that smaller, specialized models can easily match or beat giant frontier LLMs when it comes to factual accuracy.
Perplexity just rolled out a new reasoning focus mode in beta for Pro subscribers, tapping into OpenAI's o1-mini. It is built specifically for math, puzzles, and coding tasks, though it lacks web search integration for now and runs with strict rate limits.
OpenAI just boosted usage limits for its o1 reasoning models for Plus and Team subscribers. The lightweight o1-mini gets a 7x increase from 50 messages per week to 50 per day, while the costlier o1-preview bumps from 30 to 50 messages weekly. It means power users can finally test complex logic and coding tasks without constantly watching the clock.
OpenAI is cracking down on users trying to peek inside its new o1 reasoning models, sending warning emails and threatening account bans. While ChatGPT only shows a filtered summary of the model's chain of thought, researchers and red-teamers attempting to extract the raw traces—sometimes just by using the phrase "reasoning trace"—are being flagged for violating safety policies.
