Alibaba's Qwen team just dropped Qwen3.5-Omni, a native omnimodal model that handles text, image, audio, and video simultaneously. The highlight is "Audio-Visual Vibe Coding," letting you talk to your camera while the AI builds a working website or game in real time. It natively processes up to 10 hours of audio and outperforms Gemini-3.1 Pro in audio tasks.
Developer Yatharth Sharma just open-sourced LuxTTS, an ultra-fast voice cloning model that requires only a 3-second audio sample. Fitting in under 1GB of VRAM and outputting 48kHz audio, it reaches 150x real-time speed on GPUs and outpaces real-time even on basic CPUs. High-fidelity voice cloning can now run on almost any budget laptop.
Google just expanded its AI-powered Live Translate headphone feature to iOS and 12 countries across both mobile platforms. Powered by Gemini, it works with any standard pair of headphones across 70+ languages, keeping each speaker’s tone and cadence intact without requiring dedicated translation hardware.
Google just launched Gemini 3.1 Flash Live, its highest-quality audio model built for faster, more natural voice interactions. It handles complex multi-step reasoning and interruptions with ease, powers Search Live across 200+ countries, and watermarks all audio outputs with SynthID.
Cohere just dropped Transcribe, its first open-source speech recognition model designed for note-taking and voice analysis. Weighing in at 2 billion parameters, it runs on consumer GPUs, processes 525 minutes of audio per minute, and supports 14 languages. It topped the Hugging Face Open ASR leaderboard with a 5.42 WER, and Cohere is also offering free API access.
Smallest.ai just launched Lightning V3, a TTS model engineered specifically for streaming conversational agents instead of static narration. It handles 15 languages with fluid mid-sentence code-switching, accepts explicit prompts for whispering and emotional intensity, and can clone voices from just 5 to 15 seconds of raw audio. Why it matters: conversational realism requires realistic cognitive hesitation, not just robotic fluency.
Alibaba just open-sourced PrismAudio, a lightweight 518M-parameter model that generates synchronized sound effects for silent videos. By combining reinforcement learning with chain-of-thought reasoning, it generates 9 seconds of audio in just 0.63 seconds, outperforming multi-billion-parameter rivals in both speed and benchmarks.
Mistral AI just rolled out Voxtral TTS, its first text-to-speech model built for real-time voice agents. The 4B-parameter model runs at a 70ms latency, supports expressive speech across nine languages, and can clone a voice from a 3-second sample. It is available via API, in Mistral Studio, and as open weights on Hugging Face.
Late actor Val Kilmer is returning to the silver screen, with generative AI recreating his likeness and voice for the indie film As Deep as the Grave. Kilmer originally signed on to play a priest before his death in 2025, and his estate approved the AI recreation using family photos and audio under SAG guidelines. It marks one of the most prominent uses of posthumous digital casting to date.
Alibaba just open-sourced Fun-CineForge, a multimodal dubbing model that syncs speech directly to characters' lip movements and timestamps. By treating time as an explicit modality, it handles fast speaker switches and facial occlusions cleanly, turning automated film dubbing into a realistic studio workflow.
Tsinghua University researchers just open-sourced OpenMAIC, an AI platform that turns any topic or document into a full interactive classroom. Instead of watching static video lectures, users learn alongside AI teachers and classmates who speak, draw formulas on whiteboards, and engage in real-time debates. It basically shifts online learning into an active virtual classroom under a permissive MIT license.
Tencent just open-sourced Covo-Audio-Chat, a 7B end-to-end audio language model. It processes continuous audio directly in a unified architecture to enable native, low-latency full-duplex conversations that can listen, speak, and handle interruptions on the fly. Weights and code are now available on Hugging Face and GitHub.
Fish Audio just launched Fish Audio S2, an open-source TTS model trained on over 10 million hours of audio across 80 languages. Instead of tweaking global mood sliders, users can insert plain-language bracket tags like [whispering] anywhere in their script for word-level control. Model weights and fine-tuning code are open on GitHub and Hugging Face.
Researchers just open-sourced ID Laura (ID-LoRA), a unified digital human model that dumps the clunky two-step workflow of cloning audio and then syncing lips. Built on LTX-2, it generates matching video, cloned voice, and realistic ambient sound in a single pass from just an image and reference audio. Running it locally is heavy, demanding at least 24GB of VRAM.
Hume AI just open-sourced TADA, a speech-language model that runs over 5x faster than typical LLM-based TTS with zero hallucinations across 1,000+ test samples. By aligning text tokens directly to continuous acoustic vectors, it processes speech at just 2–3 frames per second and stretches a 2,048-token window to 700 seconds of audio. Why it matters: It is lightweight enough to run directly on mobile devices without relying on cloud APIs.
Tavus just launched Raven-1, a multimodal perception system that reads tone, facial expressions, and body language alongside spoken words. Instead of sorting feelings into rigid boxes like happy or sad, it generates descriptive natural language with context updated every 300ms. Why it matters: It stops conversational AI from relying on flat text transcripts, helping agents catch sarcasm and hesitation in real time.
X just launched a new Listen button for long-form articles, letting users stream AI audio voiced by Grok while scrolling or multitasking across other apps. The feature helps creators reach listeners on the go, while also generating richer long-form context to feed Grok’s training pipeline.
LTX Studio just rolled out dubbing and captions features to help enterprise teams localize video content faster. The move expands the AI video platform into multilingual corporate workflows, though details on supported languages and pricing have not been disclosed yet.
Anthropic just rolled out Voice Mode in its coding assistant Claude Code, starting with about 5% of users before expanding over the coming weeks. Eligible developers will see a note on their welcome screen and can toggle it on with /voice to steer their programming workflow with spoken words instead of typing.
Microsoft just released VibeVoice-ASR on Hugging Face, a speech-to-text model built to process up to 60 minutes of continuous audio in a single pass. By running across a 64K token window without slicing audio into chunks, it directly outputs timestamps, speaker diarization, and text across 50+ languages.
