NVIDIA's new open omni model gives AI agents a 9x speed boost

NVIDIA Nemotron 3 Nano Omni is an open multimodal model designed to give AI agents unified eyes, ears, and reasoning without juggling separate perception pipelines.

Most agentic systems today rely on a messy daisy chain: one model handles screenshots, another transcribes speech, and a core language model tries to stitch everything together. That setup bloats latency, explodes compute costs, and frequently drops critical context between inference hops. Nemotron 3 Nano Omni collapses all of those tasks into a single run.

What it does

Nemotron 3 Nano Omni handles text, high-resolution images, audio, and video inputs natively, spitting out text, chain-of-thought steps, and structured JSON tool calls.

It is targeted at five practical workloads: real-world complex document intelligence, long-form speech recognition, combined audio-video reasoning, GUI computer-use navigation, and general multimodal reasoning. The model supports documents exceeding 100 pages and accepts native 1920×1080 screen resolution for GUI tracking.

For real-time computer use agents, startups like H Company are using it to interpret full HD desktop screen recordings without waiting seconds between action steps. On audio and video, it ingests clips up to 2 minutes with 256 frames and audio up to 1 hour, weaving speech transcripts and visual cues into one reasoning stream.

Architecture, performance, and specs

Under the hood, the model uses a 30B-parameter hybrid Mamba-Transformer Mixture-of-Experts architecture that activates only 3B parameters per token. It pairs a Nemotron 3 Nano language backbone with a C-RADIOv4-H vision encoder and a Parakeet speech encoder, running a 131K context window.

In head-to-head benchmarks against alternative open omni models like Qwen3-Omni, Nemotron 3 Nano Omni scored 47.4 on the OSWorld benchmark compared to Qwen's 29.0, while topping leaderboards like MMLongBench-Doc and VoiceBench.

More importantly, system throughput scales up to 9x higher than comparable open omni alternatives under equivalent interactivity constraints, delivering 9.2x higher throughput on video workloads and 7.4x on multi-document workflows.

The weights are available now on Hugging Face in BF16, FP8, and NVFP4 formats, released alongside training recipes and datasets under the NVIDIA Open Model Agreement for commercial use. You can also run it via OpenRouter, build.nvidia.com NIM microservices, and AWS SageMaker JumpStart, with hardware deployment scaling from local Jetson units up to data center Blackwell GPUs.

Why it matters

If you are building AI agents that need to see user interfaces or process messy multimedia workflows, this is an immediate download. Passing multimodal context through a single 3B active MoE core solves the biggest bottleneck in computer use agents: perception lag.

NVIDIA is playing to its strengths here. Instead of chasing monolithic proprietary foundation models, it is giving developers a fast, open perception backbone tightly coupled to TensorRT-LLM and local GPU hardware. For enterprise teams wary of piping internal screens and audio to hosted commercial APIs, Nemotron 3 Nano Omni offers a compelling, self-hosted alternative.