Microsoft drops VibeVoice-ASR to transcribe hour-long audio in one go

VibeVoice-ASR is Microsoft's new open-source speech-to-text model on Hugging Face. Its headline trick is simple: it processes up to 60 minutes of uninterrupted audio in a single pass, spitting out clean transcripts alongside timestamps and speaker labels.

What it does

Most traditional speech pipelines slice long audio into tiny chunks, run transcription piecemeal, and stitch the results back together. That approach regularly drops conversational context and gets confused when speakers talk over one another.

VibeVoice-ASR avoids slicing altogether by operating across a 64K token window. It ingests an entire hour of continuous audio at once, keeping semantic threads intact and tracking speakers reliably from minute one to minute sixty.

The model handles speech recognition, diarization, and word-level timestamping jointly. Instead of raw text that requires downstream parsing, it outputs structured data outlining exactly who spoke, when they spoke, and what they said.

It also brings custom hotwords to the table. If you feed it technical jargon, company names, or domain-specific acronyms upfront, accuracy jumps noticeably on tricky terminology.

On top of that, it supports over 50 languages right out of the box without requiring manual language toggling, and it natively handles code-switching mid-sentence.

How to run it

Getting started is straightforward if you already live in the Python ecosystem. Support is baked directly into Hugging Face Transformers starting from version 5.3.0.

You just need to load the model and processor from the official Hugging Face repository using standard pipeline classes.

The decoding helper is particularly handy. It lets you retrieve raw text, transcription-only output, or a clean list of structured dictionaries holding speaker tags, start times, end times, and dialogue snippets.

Microsoft has shared the model weights on Hugging Face, alongside an interactive demo and a technical report covering the training details.

Why it matters

If you build meeting assistants, podcast summarizers, or interview analysis workflows, you should test this right away.

Stitching Whisper together with external diarization engines has been an awkward engineering headache for years. Slices break mid-word, timestamps drift, and identity swaps happen constantly. Microsoft tackling all three jobs inside a single monolithic model cleans up the pipeline dramatically.

That said, chewing through an hour of audio in a massive 64K token window will inevitably demand serious VRAM and compute, so plan your hosting hardware accordingly before shipping to production.