Alibaba's new omni model codes from video

Alibaba's Qwen team just rolled out Qwen3.5-Omni, a natively omnimodal model that processes text, audio, images, and video all at the same time. While some early reports referred to it as Qwen3.Omni, the release spans three distinct variants: Plus, Flash, and Light.

The headline feature is something Alibaba calls Audio-Visual Vibe Coding. Instead of typing out detailed prompts or uploading wireframe screenshots, you can simply talk to your camera, gesture, and explain your idea. Qwen3.5-Omni-Plus will then build a working website or playable mini-game right in front of you.

Long context and Gemini benchmarks

Under the hood, Qwen3.5-Omni was trained on more than 100 million hours of multimodal data. It natively digests up to 10 hours of raw audio or up to 400 seconds of 720p video in a single pass.

That capacity allows for script-level video captioning, where the model outputs full breakdowns complete with timestamps, scene transitions, and speaker mapping. It also supports speech recognition across 113 languages and can speak back in 36.

On the benchmark front, Alibaba claims Qwen3.5-Omni beats Gemini-3.1 Pro in audio tasks and matches its overall audio-visual understanding.

Polishing real-time voice and video

Beyond offline processing, the real-time capabilities got a noticeable upgrade. Real-time AI voice agents often stumble on turn-taking, interrupting users whenever there is background noise. Alibaba claims the new model features smarter turn-taking that deciphers actual conversational intent while ignoring room chatter.

Users can also tweak the voice tone, pacing, and emotional delivery on the fly. The model includes built-in web search and complex function calling out of the box, with sample-based voice cloning marked as coming soon on the engineering side.

Live demos are currently up on Hugging Face and chat.qwen.ai, alongside offline and real-time APIs on Alibaba Cloud Model Studio.

Why it matters

For builders and creators, combining visual recognition with live voice instructions changes rapid prototyping completely. Explaining a layout bug while pointing at a sketch on camera is far more natural than wrestling with prompt formatting, and testing it today on the free web chat is an easy win.

For the broader industry, this narrows the gap between open-weight ecosystem players and proprietary labs like Google and OpenAI. Matching or edging out Gemini-3.1 Pro on native audio-visual perception means high-end multimodal interaction is no longer exclusive to Silicon Valley flagships.

We think Alibaba's decision to anchor the launch around Audio-Visual Vibe Coding is a clever marketing and product move, as it grounds abstract multimodal specs in an instantly relatable developer workflow. That said, the 400-second ceiling on 720p video shows there are still hardware and context bottlenecks to overcome, and rolling out reliable real-time voice cloning at scale will bring its own set of safety and latency headaches.