Nvidia stretches robot memory to 8,000 steps with RoboTTT

Today’s robot foundation models have a goldfish memory problem. Most vision-language-action (VLA) policies are essentially myopic—they make control decisions looking only at the current frame or a tiny sliver of recent history. Ask them to perform a complex, multi-stage task, and they inevitably lose track of what they were doing, looping into failures.

Researchers from NVIDIA, Stanford, and UT Austin just introduced RoboTTT (Test-Time-Training Robot Policies), a new model and training recipe that pushes visuomotor context up to 8,000 timesteps. Running at a typical 30Hz control rate, that gives a robot roughly five minutes of continuous context—three orders of magnitude beyond current state-of-the-art policies—without blowing up inference latency.

Compressing history into fast weights

Extending context in standard Transformers usually means ballooning memory footprints and slowing inference down to a crawl. RoboTTT dodges this by ditching traditional key-value caching.

Built on top of NVIDIA's open GR00T N1.7 humanoid base model, RoboTTT inserts Test-Time Training (TTT) layers gated by learned tanh scaling. Inside each TTT layer sits a small model whose recurrent state consists of "fast weights." Instead of storing raw tokens, the model takes incoming sensor data and performs an on-the-fly gradient step using a self-supervised reconstruction loss, effectively writing recent interactions straight into its weights.

Because these fast weights are a fixed-size state carried forward, latency stays constant no matter how long the robot runs. To train across sequences that long without melting GPUs, the team combined truncated backpropagation through time (TBPTT)—propagating gradients only within fixed segments while letting fast weights flow across them—with a technique called sequence action forcing, which applies independent noise levels across action chunks to stabilize flow matching.

Zero-shot video imitation and online recovery

The jump from seconds of memory to minutes unlocks behaviors that short-context policies simply cannot replicate.

On real-world dual-arm manipulation setups, RoboTTT improved overall performance by 87% over single-step baselines. In a grueling five-minute, 10-stage circuit assembly challenge, none of the baseline policies could finish; RoboTTT completed the full task. Performance scaled steadily with context length: models trained with 8K context outperformed those with 1K context by 62%, showing no signs of plateauing.

Even cooler are the emergent downstream skills. RoboTTT can pull off one-shot in-context imitation: feed it a single video of a human demonstrating a novel assembly as pure context (by masking action losses on those frames), and the robot figures out where unfamiliar components go using only a generic instruction. It also demonstrated perturbation robustness and online self-correction via DAgger distillation. When researchers physically stole an already-installed component mid-run, the robot referenced its own history, doubled back, and reinstalled it.

Why it matters

Until now, scaling in robotics has mostly been a two-horse race: more model parameters and larger datasets of real-world trajectories. RoboTTT convincingly demonstrates that context length could be the third scaling axis for robot foundation models.

For roboticists, long-horizon dexterity has always required clunky prompt-chaining or fragile state machines to stitch sub-tasks together. Handing a robot five minutes of working memory makes autonomous, multi-phase assembly and chore execution far more viable right out of the box.

We think using TTT to compress trajectories into weights is exceptionally clever because high-frequency robot control (30Hz or faster) cannot tolerate the quadratic latency of Transformer attention. The trade-off, however, lies in running continuous gradient updates directly on inference hardware, which adds software complexity and edge-compute demands. Still, giving robots an active working memory instead of making them guess from scratch every few milliseconds is a major leap toward truly autonomous agents.