Published: 2026-10-30
Summary
Summary
Qwen Audio Agent as a voice front end for Claude Code: install, config.env, gateway, and what interruption and cancel really do
Chapters / key moments (click to jump — plays here on the page)
The idea: keep talking to a voice assistant while Claude Code builds in the background. A local gateway connects a Qwen realtime voice model to Claude Code, so short questions are answered by the voice model and coding work goes to Claude Code. The video's value is in the setup details and the honest tests of what voice control does and does not do.
Source video
"Qwen Audio Agent + Claude Code: This Basically makes Claude Code a HUMAN!" by AICodeKing — Watch on YouTube → The automatic captions spell the tool's name inconsistently, so we do not reproduce its command lines. The speaker says the exact commands are on screen and in the video description; use the project's README.
Setup, in order
- Runtime. Node 24 series, starting at 24.15, with npm 10 or newer (check with
node --versionandnpm --version). A desktop installer exists and bundles its own runtime; the video uses the command-line install and the browser interface. - Install version 2.0.1 globally with npm. The video's instructions pin that version.
- Open the config. A config command prints the path to the configuration file. The default is
config.envin a folder named after the tool under.configin your home directory. - Voice provider. Create an API key in Alibaba's DashScope console and put it in the DashScope API key field. Set the realtime provider to DashScope and use the Qwen audio 3.0 realtime plus model shown in the config. The software is open source but the cloud voice connection needs an API account, and Claude usage is billed separately.
- Claude components. An install-Claude command checks for Claude and installs missing pieces, including the adapter for the Agent Client Protocol (ACP). If you already use Claude Code it can reuse that installation.
- Check Claude on its own. Run
claude, sign in or configure API access, and make sure a small request works before debugging the voice layer. Then run the tool's setup command with the Claude back end; it checks the integration components but does not prove your account has quota. - Config fields. Set the agent protocol to
claude; leave the back-end model empty to use Claude's own model configuration; set the back-end permission mode tonativeso Claude's normal permission behaviour applies; set the workspace to the full absolute path of a separate project folder. - Start it. Run the gateway and keep that terminal open; in a second terminal start the web UI. The default local address ends in port
3101. Allow microphone access, enable the mic, and confirm the gateway and voice connection show as connected. Use headphones for the first test.
Tests the speaker ran
- Transcript check. Ask for a short English reply and check the on-screen transcript as well as the audio. If the words are wrong, Claude receives the wrong request.
- Workspace check. Ask Claude to report the absolute path of its workspace and list the files without changing anything; compare with the folder you chose.
- Main build. A small "voice desk" issue tracker in plain HTML, CSS and JavaScript with a dark theme, add and complete buttons, a completed filter, local-storage persistence and basic validation. Kept small so a misrouted correction is easy to spot.
- Talking while it builds. A separate question (what does local storage do here) was answered by the voice model while the Claude task kept running.
- Interrupting speech. Telling it to make an answer shorter stops the spoken reply, but does not cancel a task the back end already accepted.
- Mid-build change. Asking for a light theme and a renamed button was accepted as a follow-up. It may become subsequent work rather than altering the current turn, so check task state and the files, not the spoken acknowledgement.
- Debug request. Spaces-only titles: ask Claude to explain how the code handles it, add a regression check and fix it while preserving existing saved issues, then test empty, spaces-only and normal titles and refresh to confirm old data survives.
Gotchas
- Terminal location is not the workspace. Opening a terminal in a folder does not make it the agent's workspace; the app has its own default, and a Claude-specific workspace setting can override the general one. Check the configured path.
- New session, not your existing one. It opens a new agent workflow; it will not take over a conversation already open in another Claude terminal.
- Permissions. Allowing a task can also allow subsequent operations within that task. The speaker used a disposable project.
- Cancel is separate from stop talking. Test cancellation with a read-only task (an accessibility review with no edits). A request to cancel and a completed cancellation are different events, cancelling does not undo edits already made, and muting the mic or closing the conversation does not stop background work. Use the task cancel control.
- Use text for exact things. File paths, code and error messages are better pasted than spoken; ask for launch instructions as copyable text.
- Two sources of usage. DashScope for the voice model and your Claude account or API for the coding work. Check both providers' current prices for your region and record both sides of the same session if comparing costs.





