Nvidia's new 9B model lets you toggle reasoning on and off
Nvidia just dropped Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer model slimmed down to fit on a single A10 GPU. Developers can toggle reasoning traces on or off using simple /think tokens and dial in exact thinking budgets. It delivers up to 6x faster inference than similar transformers while outperforming Qwen3-8B across key benchmarks.
Nemotron-Nano-9B-v2 is an open-source text-only small language model from Nvidia that brings heavy-hitting reasoning down to a single GPU footprint. It gives developers full control over internal chain-of-thought processing, letting you toggle reasoning on and off on the fly.
Turn reasoning on with a slash command
Most reasoning models force you into a one-size-fits-all mode: they either burn extra compute overthinking simple queries or lack deep reasoning capabilities entirely. Nemotron-Nano-9B-v2 defaults to generating a self-checking reasoning trace before answering, but you can override this anytime using simple control tokens like /think or /no_think.
Even better, Nvidia baked in a runtime thinking budget feature. Developers can set an exact cap on how many tokens the model spends on internal reasoning before completing its output. That lets you balance accuracy against latency down to the millisecond, which is huge for live agent workflows and customer-facing bots.
Under the hood, the model belongs to Nvidia's Nemotron-H family, combining Mamba selective state space models (SSMs) with standard Transformer attention layers. Because the SSM layers scale linearly with sequence length instead of quadratically, the hybrid design yields 2x to 3x higher throughput on long contexts and runs up to 6x faster than pure Transformer models of similar size while supporting larger batch sizes.
Tailored for an A10, outperforming Qwen3-8B
Nvidia deliberately pruned the model from an original 12 billion parameters down to 9 billion specifically so it runs comfortably on a single Nvidia A10 GPU, one of the most widely deployed workhorse cards in cloud environments.
Despite the lean footprint, the benchmark results punch well above its weight class. Tested with reasoning enabled on the NeMo-Skills suite, Nano-9B-v2 hit 72.1% on AIME25, 97.8% on MATH500, 64.0% on GPQA, and 71.1% on LiveCodeBench. It also scored 90.3% on IFEval and 78.9% on the RULER 128K context benchmark, consistently outperforming Qwen3-8B across the board.
The model handles English, German, Spanish, French, Italian, and Japanese, with extended coverage for Chinese, Korean, Portuguese, and Russian. Both the model weights and pre-training datasets are live on Hugging Face and Nvidia's model catalog under the permissive Nvidia Open Model License Agreement updated in June 2025.
Why it matters
If you are building autonomous agents or customer service pipelines on a budget, this is something you should test immediately. The ability to treat reasoning depth as a dial rather than a fixed tax solves a massive headache in real-world deployments.
It also proves that the hybrid Mamba-Transformer architecture is ready for enterprise workloads. Instead of blindly chasing 70B-plus parameter counts, Nvidia designed around practical hardware constraints and gave developers the granular knobs they actually need.
Sources (1)
- venturebeat.com(opens in a new tab)Originalventurebeat.com