Alibaba open-sources real-time voice runtime for AI agents

qwen-audio-agent is an open-source real-time voice runtime developed by Alibaba that basically gives your coding and task agents ears and a mouth. It sits as a full-duplex conversational voice frontend while delegating deep, long-running tasks to backend agents, letting you collaborate with AI just like being on a phone call.

The biggest friction with traditional voice assistants is that the moment they need to write code, look up docs, or run a tool, the entire conversation halts and leaves you waiting in silence. qwen-audio-agent separates the conversational layer from task execution, running them in parallel so your conversation stays live while work gets done in the background.

What it does

The frontend handles full-duplex audio, which means you can talk naturally, ramble through an idea, and interrupt whenever you want without pressing buttons or waiting for a turn to finish.

When you assign a heavy engineering task, such as refactoring a module, running tests, or searching a repository, it delegates the job to a backend agent. The backend runs asynchronously while the voice session continues uninterrupted. You can check on progress or cancel mid-way by just asking out loud.

Once the task completes, the agent jumps back into the call with a natural update like "It's ready" and brings the output into the active context. If your voice model supports vision, it can also ingest camera streams for real-time visual assistance.

It offers multiple client interfaces, including a WebUI, a terminal TUI, and a desktop floating orb for macOS, Windows, and Linux. The desktop version comes with built-in gateway services, customizable skins, local wake-word support with "Nihao Qianwen", and persistent cross-session memory.

How to get started

The architecture is heavily decoupled. You can independently swap the voice frontend and the backend execution agent based on your needs.

For the voice layer, it defaults to Qwen Audio 3.1 Realtime Plus via Bailian, but it also supports OpenAI Realtime, Google Gemini Live, Doubao, StepAudio, and MiniCPM-o. For local enthusiasts, it supports self-hosted Hugging Face pipelines with custom VAD, STT, LLM, and TTS setups.

On the backend side, it connects through standard protocols like ACP to popular agent runtimes including OpenCode, OpenClaw, Claude Code, Codex, Qoder, Kimi Code, and DeepSeek Harness. You keep your existing agent configs, MCPs, and authentication without rebuilding anything from scratch.

Running it requires Node.js 22.22.2+ or 24.15.0+ and npm 10+. You can install it globally via npm, configure your DashScope API key, and launch the gateway followed by the TUI or WebUI. Prebuilt desktop installers for Mac, Windows, and Linux are also hosted on GitHub Releases.

Why it matters

If you use coding agents daily, this project is definitely worth trying. It breaks away from typing out lengthy prompts and staring at terminal spinners, shifting the workflow into continuous voice brainstorming while tasks run quietly in the background.

Compared to walled-garden setups like GPT-Live, Alibaba took an aggregator approach. It acts as an open harness that bridges arbitrary real-time voice models with arbitrary execution backends. We think decoupling conversation from execution is a clever structural move that solves the awkward latency gaps in heavy agent workflows, offering a practical voice frontend standard for the open-source ecosystem.