Reka collapses multimodal stacks with 19B omni-reasoning model Rho-1

If you want an AI to paint a lighthouse, draw a bounding box around it, animate that scene into video, change the weather to a blizzard, and then explain the subtle visual differences between the two clips, you usually need a chaotic pipeline of four or five separate models.

Each handoff between those specialized tools introduces latency, and every model in the chain sees only a narrow slice of context rather than the full scene history.

Reka, an AI lab founded by former DeepMind researchers, just released a research preview of Rho-1. It is a 19B omni-reasoning model trained from scratch that collapses that entire multimodal stack into a single neural network.

Collapsing the stack into one context

Instead of bolting an LLM planner onto dedicated diffusion systems, Rho-1 treats text, visual tokens, and robotic actions symmetrically within a single context window.

Inside each transformer block, the model runs two expert streams. An understanding stream handles linguistic reasoning and scene parsing, while a generation stream denoises latent representations into images and video using flow matching.

Crucially, both streams share the exact same attention space and key-value cache. When Rho-1 generates an image, that visual representation remains live in memory.

When you ask the model to crop an object, animate the frame, or modify the weather, it does not re-encode exported pixels. It reads directly from the persistent world state, eliminating pipeline latency.

Rho-1 is remarkably fast. The base model renders video at roughly 0.79x real-time speed, with playback starting in about 6 seconds. A distilled variant slashes the denoising trajectory from 99 steps down to 8, generating a 5.3-second video clip in about one second.

Beyond video, Rho-1 natively decodes motor policies alongside visual frames. The same network predicting the next scene observation directly dictates robotic joint actuation, unlocking an integrated foundation for embodied AI.

There are clear research-stage limits. Video resolution is capped at 672x384, long rollouts past 30 seconds suffer structural drift, and fine-grained visual editing across diverse prompts remains brittle.

Why it matters

Rho-1 points toward an architectural shift that challenges the prevailing trend of chaining specialized models together.

What makes this release remarkable is the compute efficiency behind it. Reka trained this 19B model from scratch using just 320 Nvidia H100 GPUs over three months. Demonstrating real-time, unified text-image-video-action capabilities on such a modest training budget is an impressive feat of engineering.

For developers building interactive games, robotic simulations, or responsive video tools, a persistent world state that you can interrogate and edit mid-stream is far more practical than waiting on brittle multistep pipelines.

We think this native unification is the right direction for embodied intelligence. When language models act merely as dispatchers for separate vision and video engines, grounding gets lost in translation. Rho-1 proves that letting reasoning and generation share the same latent space allows physical intuition and world simulation to reinforce each other, even if scaling it up to frontier resolution remains an open challenge.