Alibaba open-sources Qwen3-Omni with real-time audio and video

Qwen3-Omni is Alibaba's open-source, natively end-to-end multimodal model that integrates text, images, audio, and video under a single neural architecture.

Instead of chaining separate speech recognition, text inference, and speech synthesis pipelines, it natively ingests multiple modalities at once and generates speech and text directly.

What it can do

The most striking metric is speed: Alibaba clocked its response latency at 211ms, matching the pacing of natural human dialogue.

It handles up to 30 minutes of continuous audio input, making it capable of analyzing lengthy meetings, recordings, and combined audiovisual files. Alibaba notes that the model achieved state-of-the-art results on 22 out of 36 audio and audiovisual benchmarks.

Language support spans 119 languages for text input, 19 languages for speech input, and 10 languages for direct speech output. The system also includes native tool-calling capabilities and supports full persona and behavioral customization via system prompts.

Available weights and where to try it

Alibaba published the project on GitHub and uploaded model weights to both Hugging Face and ModelScope collections.

The released lineup consists of 30B-scale models across three configurations: Qwen3-Omni-30B-A3B-Instruct for standard conversational use, a Thinking variant designed for complex reasoning, and a specialized low-hallucination Captioner model. Platform listings place the parameter footprints between 32B and 35B.

For those wanting to test it without local setup, interactive spaces are live on Hugging Face Spaces, and a flash version is integrated directly into Qwen Chat. Deploying locally requires substantial GPU VRAM to handle real-time streaming buffers alongside the 30B multimodal core.

Why it matters

For developers building real-time voice assistants, customer agents, or embodied robotics, Qwen3-Omni is one of the most capable open foundations available right now.

Real-time, end-to-end multimodal execution has largely been restricted to closed commercial services. Delivering an open 30B-scale model—alongside a dedicated Thinking reasoning checkpoint—gives builders genuine control over low-latency interactive agents.

Keep in mind that serving continuous audio and video streams requires far more infrastructure orchestration than typical text-based LLMs. We recommend testing real-world acoustic conditions and latency in the hosted demos before spinning up dedicated GPU hardware.