Monarch RT delivers 16 FPS real-time video on a single RTX 5090

Until now, generating video with AI has meant staring at rendering progress bars or paying for massive cloud clusters. The idea of doing live, real-time video generation on a local workstation sounded like science fiction.

Researchers introduced Monarch RT, a structured attention architecture that brings genuine real-time video diffusion down to a single consumer graphics card. Running on an Nvidia RTX 5090, it generates video at 16 FPS.

The complete code and local setup instructions are already live on GitHub.

Cracking the 3D attention bottleneck

The fundamental bottleneck in Diffusion Transformers is the quadratic cost of 3D spatiotemporal self-attention. In real-time regimes with few steps and autoregressive rollouts, errors compound rapidly, and prior sparse-attention tricks simply fall apart under the pressure.

The researchers realized video attention is not randomly sparse. Instead, it blends periodic patterns driven by spatiotemporal positions with dynamic semantic correspondences.

To capture that structure, Monarch RT factorizes video attention using Monarch matrices. Combined with a tiled parameterization and custom-tuned Triton kernels, it achieves up to 95% attention sparsity without sacrificing model expressivity.

Up to 4x faster without visual collapse

When paired with the state-of-the-art Self-Forcing video model, Monarch RT delivers sustained 16 FPS generation on a single RTX 5090.

Its attention mechanism runs nearly 4x faster than standard FlashAttention. Across RTX 5090, H100, and B200 GPUs, benchmarked against FlashAttention-2, FlashAttention-3, and FlashAttention-4, Monarch RT clocked kernel speedups between 1.4x and 11.8x.

Critically, the speed jump does not destroy video quality. Under identical prompts, Monarch RT retains roughly 95% of full-dense attention quality, with virtually imperceptible differences in side-by-side previews.

Why it matters

We think Monarch RT marks the moment real-time AI video transitions from a data-center luxury into an accessible developer tool.

An RTX 5090 is still an expensive, top-tier consumer card, but owning a single GPU is infinitely more practical than renting an enterprise cluster. For game developers, live streamers, and creative coders, local real-time generation is now physically within reach.

Technically, factorizing spatiotemporal attention with Monarch matrices was a remarkably clever bet. Instead of aggressively pruning tokens and risking flickering artifacts, the team addressed the mathematical geometry of video attention directly.

This will likely push competing video model labs to adopt similar structured matrix kernels. The remaining hurdle is expanding this efficiency to mid-tier GPUs, but single-card 16 FPS already proves the real-time era of video diffusion has officially arrived.