NVIDIA's new embeddings top RTEB, cutting agent token waste

NVIDIA’s Nemotron 3 Embed is an open, commercially usable embedding collection designed specifically for production RAG and agentic workflows. It includes three models: a flagship 8B variant that takes the top spot on the RTEB multilingual benchmark, along with two compact 1.14B variants engineered for latency-sensitive deployments.

The core goal here is fixing agent inefficiency. In multi-step agentic systems, weak retrieval forces agents to fetch irrelevant context, launch repetitive queries, and waste token budgets on unnecessary reasoning cycles.

What it does

All three models support a 32k context window. That means you can feed in long enterprise documentation, large multi-file codebases, and long agent conversation logs without suffering aggressive text truncation.

The models use standard mean pooling and simple input formatting: just prepend inputs with query: or document: tags. They natively handle multilingual enterprise data as well as technical code retrieval.

In benchmark evaluations, Nemotron-3-Embed-8B-BF16 scored 78.5% on RTEB and 75.5% on MMTEB Retrieval, ranking first overall. The compact 1.14B BF16 version reached 72.4% on RTEB, slashing error rates by 27% compared to its previous-generation 1B predecessor.

Crucially, NVIDIA evaluated how these embeddings impact downstream agents. By surfacing accurate evidence on the first attempt, the models help search agents complete tasks with fewer iterative steps, directly reducing downstream token consumption.

How it works

Instead of training small embeddings from scratch, NVIDIA built these models by adapting and compressing strong foundation models.

The 8B flagship adapts the Ministral-3-8B-Instruct-2512 backbone, turning its causal decoder into a bidirectional encoder for full-sequence retrieval. It was pre-trained with contrastive learning and fine-tuned on curated domain datasets spanning legal, finance, medical, and technical corpora.

To build the 1.14B version, NVIDIA started with a 3B bidirectional base derived from Ministral-3-3B. The team used NVIDIA ModelOpt neural architecture search to prune parameters down to 2B, then distilled knowledge from the 8B teacher model to restore ranking quality.

They repeated this pruning and distillation cycle a second time to hit the final 1.14B footprint, followed by a progressive training schedule that stretched the context window from 1024 to 4096 tokens.

For enterprise infrastructure, NVIDIA also introduced Nemotron-3-Embed-1B-NVFP4. Optimized specifically for the NVIDIA Blackwell architecture, it uses native 4-bit linear layers and Quantization-Aware Distillation to achieve up to 2x higher throughput than BF16 while retaining over 99% of retrieval accuracy.

How to run it

Weights, datasets, and distillation recipes are available immediately on Hugging Face under commercial-friendly terms.

You can run the models locally using vLLM or deploy them via NVIDIA NIM containerized microservices. For teams wanting to adapt the embeddings to proprietary internal data, NVIDIA NeMo AutoModel recipes provide end-to-end fine-tuning workflows.

In terms of hardware, the 8B-BF16 variant runs on standard inference GPUs when maximum retrieval accuracy is required. The 1.14B-BF16 variant is light enough for low-latency CPU or entry-level GPU inference. If you have access to Blackwell or GB200 systems, the NVFP4 variant is the obvious choice for high-volume pipelines.

Why it matters

If you run production RAG systems, code search tools, or multi-turn agent pipelines, this release is worth testing right away. Upgrading your embedding model is often the cheapest performance gain in an agent stack, because cleaner context immediately prevents expensive LLMs from burning tokens on useless reasoning turns.

We think NVIDIA played this smartly. The 8B model takes the benchmark crown to prove quality, while the 1B models deliver practical, cost-effective inference for real-world traffic. You do need Blackwell hardware to exploit the NVFP4 throughput gains, but even on standard hardware, the 1.14B BF16 checkpoint offers a compelling balance of speed and precision.