NVIDIA unveils Rubin CPX to power 1M+ token AI inference

Generating high-definition video or having an AI review an entire code repository can chew through compute at an alarming rate. When running massive workloads, the biggest headache usually isn't raw compute power, but the latency and energy wasted moving data across separate chips.

NVIDIA unveiled Rubin CPX, a new chip system built from the ground up for heavy-duty AI inference, specifically targeting workloads that demand more than 1 million tokens of context.

Packing the entire pipeline onto one chip

Traditional video and complex reasoning pipelines typically lean on multi-chip setups. One chip handles video decoding, another runs the actual model inference, and yet another encodes the finished output.

That back-and-forth shuffling creates real bottlenecks, slowing down throughput and introducing unwanted hardware inefficiencies.

Rubin CPX takes a much cleaner approach by integrating video decoding, encoding, and inference onto a single unified platform.

NVIDIA is aiming this system directly at two compute-hungry use cases: AI video generation and software development. Both domains routinely demand sustained processing power capable of handling up to a million tokens per hour or enormous continuous context windows.

By collapsing the media pipeline and the inference engine into one roof, the chip cuts out multi-chip communication overhead entirely.

Why it matters

For developers and platforms building long-context agents and video generation tools, inference efficiency is make-or-break. Running million-token prompts on codebases or rendering high-res video clips eats margins fast. A unified chip system that handles encoding, processing, and decoding in one go means lower latency and simpler infrastructure.

Across the industry, this highlights how quickly hardware competition is shifting from pure model training toward specialized, high-volume inference. While competitors race to build basic AI accelerators, NVIDIA is tackling the practical media bottlenecks that bottleneck real-world generative video pipelines.

We think this is a smart architectural pivot. Consolidating fragmented steps into a single platform solves real production headaches. That said, whether it truly transforms deployment economics will ultimately depend on real-world energy efficiency and where NVIDIA sets the final price tag.