NVIDIA drops Cosmos to simulate billions of driving miles

NVIDIA Cosmos is an open platform of physical AI world foundation models built for autonomous vehicles and robotics. It bundles generative foundation models, advanced tokenizers, and an accelerated video pipeline to turn a few thousand human-driven miles into billions of synthetic driving miles.

Inside NVIDIA's three-computer AV stack, Cosmos acts as a data amplifier. Paired with the Omniverse simulation platform, developers can procedurally generate photorealistic edge cases, variations, and traffic flows for closed-loop testing without burning millions on real-world test fleets. Teams like Waabi and Wayve are already using it to mine corner cases, while Uber is pooling its massive driving data to feed the loop.

Architecture and hardware requirements

Cosmos 3 runs on a unified Mixture-of-Transformers architecture split into two towers. The Reasoner takes text and visual inputs to handle embodied reasoning and task planning, while the Generator ingests multimodal inputs—text, vision, audio, and actions—to simulate dynamic worlds and predict action policies.

NVIDIA ships the family in three tiers based on hardware targets.

Cosmos3-Super (64B) is the heavy hitter, requiring H200 or B200 clusters to produce top-tier synthetic data and serve as a teacher model for distillation.

Cosmos3-Nano (16B) strikes the balance between speed and quality, running on a single workstation-grade RTX Pro 6000, H100, or B200, making it the practical workhorse for fine-tuning.

Cosmos3-Edge (4B) is designed for on-device deployment, fitting into embedded hardware like Jetson AGX Orin or Drive Thor for real-time robot policies.

How to run it

Cosmos models are hosted under an open model license on Hugging Face and NVIDIA NGC, with code and recipes open-sourced on GitHub. NVIDIA NIM microservices are also on the roadmap.

Running it locally requires compute and a gated permission step. Before loading the diffusers pipeline, you must request access to the Cosmos-1.0-Guardrail repo on Hugging Face and authenticate with a read token.

The repository features a comprehensive Cosmos Cookbook, complete with data curation tools, post-training recipes, and serving guides for vLLM. Keep in mind that running the default Nano pipeline pulls down a 16B checkpoint, and diffusion sampling requires considerable VRAM and compute time.

Why it matters

For autonomous driving engineers and robotics researchers, Cosmos democratizes the kind of generative world simulation pipeline that previously required tens of millions of dollars to build in-house. It closes the synthetic data loop with production-ready tooling that teams can actually deploy and adapt.

That said, it is not a casual weekend tool for solo hobbyists. Even the 16B Nano tier requires beefy enterprise GPUs to run comfortably, and the gated guardrail requirement means setup takes some paperwork. It is a serious infrastructure foundation meant for production physical AI stacks.