NetEase Youdao open-sources R2T2 for flicker-free speech transcription

Confucius4-R2T2 (or simply R2T2) is an open-source real-time streaming speech recognition model from NetEase Youdao that turns spoken audio into text with virtually zero lag. If you are building live transcription tools or voice agents and are tired of words flickering and rewriting themselves mid-sentence, this tool was built for you.

What it actually does

R2T2 operates in an append-only mode: once a word is committed, it never changes. Traditional pseudo-streaming models frequently spit out an early guess and overwrite it a split second later as more audio trickles in, causing annoying screen flicker and breaking downstream LLM pipelines.

R2T2 avoids that rollback headache completely. It delivers an average latency between 200 and 600 ms, with configurable decoding chunk sizes ranging from 80 ms to 2 seconds (160 ms by default).

Even better, switching to streaming mode does not sacrifice accuracy. Its transcription quality stays virtually identical to full offline recognition.

How it works and how it performs

Built on top of Qwen3-ASR, R2T2 is trained using stable-prefix data, forced time-alignment, and token-level audio segmentation, tied together with what Youdao calls a Longest Stable Prefix (LSP) learning paradigm.

In plain English, the model dynamically decides whether the current audio context is solid enough to emit permanent text or whether it should hold back for another fraction of a second. This ensures committed text stays untouched while providing clean context for subsequent tokens.

On standard English WER and Chinese CER benchmarks, R2T2 lands on the most optimal spot of the accuracy-latency Pareto frontier among open-source models, outperforming alternatives like WhisperRT, Nemotron, and Voxtral while keeping pace with proprietary options like GPT-Live-Transcribe. It also natively supports context and hotword prompting, optimized for Chinese and English with broad multilingual coverage.

How to run it

The hardware barrier here is surprisingly low. The model checkpoint takes up around 4GB, meaning it comfortably runs on most consumer GPUs.

It supports both a high-throughput vLLM backend and standard Hugging Face transformers. Youdao recommends running it via Docker using the official Qwen3-ASR image, though you can also install it natively with Python 3.12 using uv or Conda.

The repository includes ready-to-use Python snippets, CLI entry points, and a full WebSocket streaming server script paired with voice activity detection (VAD) for multi-client production setups.

Why it matters

For anyone building real-time captioning, simultaneous interpretation, or voice-first AI agents, R2T2 is an easy recommendation. It eliminates transcript churn while keeping latency under half a second on consumer-grade silicon.

We think framing streaming output as an immutable append-only event stream is the smartest architectural choice here. Downstream LLMs finally receive predictable inputs without complex rollback logic. Keep in mind that Youdao has yet to publish the full technical report, so real-world robustness across noisy environments and niche languages remains to be battle-tested.