Talk to your laptop. Watch Claude Code write the code.

License: MIT Platform: macOS | Linux Works with Claude Code

yapcode is one voice agent for all your Claude Code sessions. It becomes Claude's mouth and ears: you speak, and it drives real Claude Code sessions on your machine via tmux — starting them across your projects, sending instructions, approving permission prompts, running slash commands, and narrating the results back. A live terminal streams the actual Claude TUI to your browser and your phone, so you can watch (and take over by keyboard) at any time.

Watch the demo

▶ Watch the demo

• 🎙️ Hands-free Claude Code — speak; the agent drives a real claude session and reads the answer back.
• 🖥️ Live terminal, anywhere — the Claude TUI streams to browser and phone via xterm.js; watch it work or grab the keyboard.
• 🔀 OpenAI Realtime or Gemini Live — pick your voice model (OpenAI direct or via Azure; Gemini has a free tier). Keys stay server-side.
• 📱 Code from mobile (on your local network, for now) — talk to Claude and watch the live terminal from anywhere in the house. Setup: use it from your phone.
• ✅ Approve by voice — say "yes", "allow", or "switch to auto mode" without touching the keyboard.
• 🤝 Co-drive — voice and keyboard share one tmux session; type and talk on the same session simultaneously.
• 🔁 Hand off in either direction — /voice-handoff continues a terminal session by voice; the printed tmux attach takes the keyboard on a voice one. Details: co-driving.
• 🖥️ Control the real TUI via voice — "run /init" · "interrupt that" · "what's on the screen?"
• 🗂️ Juggle projects — list, start, rename, and close sessions across your allowed folders, by name, entirely by voice.

yapcode up checks these and tells you what's missing (full list):

• Claude Code, installed and logged in
• Voice API key — Gemini (free tier), OpenAI native, or OpenAI via Azure
• tmux
• Python 3.12+
• Node 20+

git clone https://github.com/nithiink/yapcode.git
cd yapcode
./bin/yapcode up # first run: setup wizard → installs deps → opens http://localhost:3000

Don't want to install the prerequisites yourself? Use Homebrew — brew pulls in tmux, python@3.12, and node for you:

brew tap nithiink/yapcode
brew trust nithiink/yapcode # one-time: newer brew gates third-party taps until trusted
brew install yapcode
yapcode up # same wizard; opens http://localhost:3000

The first-run wizard asks for a voice key and the folder(s) the agent may edit; after that, yapcode up just launches.

• How it works
• Security model (read this first)
• What's in the box
• Prerequisites
• Install from source
• Install with Homebrew
• Running
• Use it from your phone (local network)
• Windows (WSL2)
• Configuration reference
• Security & access control (details)
• Co-driving: voice + keyboard on one session
• Troubleshooting
• Architecture (for contributors)
• Contributing
• License

 you ──speak──▶ Voice agent (OpenAI / Azure, or Gemini Live)
 │ tool calls
 ▼
 Backend (FastAPI, :8000)
 │ tmux send-keys / capture-pane
 ▼
 Claude Code CLI (real `claude` session in a tmux pane)
 │
 ▼
 browser/phone ◀── live xterm.js terminal (Next.js frontend, :3000)

• Backend (backend/) — FastAPI. Mints short-lived voice-provider tokens (your provider keys stay server-side; the browser only ever gets an ephemeral token) and turns the voice agent's tool calls into real actions against Claude Code via tmux.
• Frontend (frontend/) — Next.js 16 / React 19. The browser UI, the realtime audio transports, and the live xterm.js terminal that streams the Claude TUI.
• Each session runs in a tmux session named vc_<id>, so the voice agent, the browser terminal, and your own keyboard can all drive the same Claude session at once.

The code-level path through this pipeline is in Architecture (for contributors).

yapcode runs a backend that executes commands on your machine. It has two independent layers of defense — both fail-closed.

In plain language:

• On localhost, it just works with zero config — the backend trusts loopback (your own machine: 127.0.0.1, ::1, localhost), so no token is needed.
• The moment you go off your laptop (LAN, phone, tunnel), you must set a token, and the network launcher will not start without one.
• Sessions only start inside folders you allowlist (ALLOWED_PROJECT_ROOTS), and the voice agent itself can't run arbitrary commands — its only tools spawn and drive Claude sessions. What Claude does inside a session is governed by Claude Code's own permission model: in default mode it asks first and you approve or deny (by voice or keyboard); auto mode acts without asking, exactly like at the keyboard.

Full details — the _access_ok check, Origin allowlist, CSRF defenses, log redaction — are in Security & access control (details) below.

yapcode/
├── backend/ # FastAPI: mints voice tokens, drives Claude via tmux
├── frontend/ # Next.js 16 / React 19: UI, voice transports, xterm terminal
├── integrations/ # Claude Code plugin (/voice-handoff co-driving)
│ └── claude-code-plugin/
├── packaging/ # Homebrew formula + release script
│ ├── yapcode.rb
│ ├── release.sh
│ └── README.md
├── bin/ # the `yapcode` CLI launcher (bash)
├── LICENSE # MIT
└── README.md

# macOS — installs everything except Claude Code itself
brew install tmux node python@3.12 git

On Debian / Ubuntu, apt covers tmux/Python/git, but its Node is usually too old — install Node 20+ via nvm (as shown in the WSL2 section), not from apt. Use Ubuntu 24.04+ for Python 3.12.

# Debian / Ubuntu — Python/tmux/git only (install Node 20+ separately via nvm)
sudo apt install tmux git python3 python3-venv

This is the primary path — git clone, then ./bin/yapcode up. The commands are in the quickstart; the details unique to a clone:

• On first up the launcher bootstraps dependencies: if backend/.venv is missing it picks a Python 3.12+ interpreter and pip installs backend/requirements.txt; if frontend/node_modules is missing it runs npm ci. (It does not install tmux, Python, or Node — those are prerequisites; use Homebrew if you'd rather not install them yourself.)
• It then preflights ports 8000/3000, starts both servers, polls for readiness (~60s), prints yapcode is running — http://localhost:3000, and opens it.
• ./bin/yapcode config and ./bin/yapcode session work from a clone too.

Use this if you want full control of each piece (and for contributing — see Contributing).

# Backend
cd backend
python3 -m venv .venv # MUST be Python 3.12+ (see Troubleshooting)
.venv/bin/pip install -r requirements.txt
cp .env.example .env # then edit: at minimum a voice provider key
 # and ALLOWED_PROJECT_ROOTS
# Frontend
cd ../frontend
npm ci

ALLOWED_PROJECT_ROOTS and a voice provider key are the minimum required config. tmux must be on your PATH. On macOS, if your default python3 is older than 3.12:

brew install python@3.12
# then create the venv with python3.12 -m venv .venv

One config file. In a clone the single source of truth is backend/.env — the wizard writes it, yapcode config edits it, and the backend loads it directly, so there's no second location and no precedence to track. (Only a Homebrew install differs: its read-only Cellar can't hold a writable backend/.env, so the wrapper sets YAPCODE_CONFIG_DIR and the file lives at ~/.config/yapcode/.env instead. You can set that same variable in a clone if you'd rather keep config outside the tree.)

The first time you run any subcommand (up / session / config) with no config present, a wizard runs and writes the config file (created 0600, umask 077) — backend/.env from a clone, or ~/.config/yapcode/.env on a Homebrew install. It prompts for:

1. Gemini API key — optional (free tier at https://aistudio.google.com/apikey), Enter to skip.
2. OpenAI key (sk-...) — optional, Enter to skip.
3. Folder(s) the agent may edit — comma-separated; a leading ~ is expanded. At least one is required; each must exist and may not be empty, /, or your $HOME (rejected as too broad).

The wizard derives the default VOICE_PROVIDER (Gemini key → gemini; else OpenAI key → openai; else openai placeholder with a warning to add a key later) and auto-generates a VC_AUTH_TOKEN for later network/phone use. Azure OpenAI is config-file-only (not prompted) — add it later with yapcode config.

Only the absence of the config file triggers the wizard; later runs never re-prompt. To change anything, edit the file or run yapcode config.

yapcode -h / --help prints usage: yapcode {up|session [dir]|config}. An unknown subcommand prints usage to stderr and exits 2.

Prefer not to install the prerequisites yourself? brew install pulls in tmux, python@3.12, and node and builds the app — handy on a fresh machine. Works on macOS and Linuxbrew:

brew tap nithiink/yapcode
brew trust nithiink/yapcode # one-time; see note below
brew install yapcode # pulls in tmux, python@3.12, node; builds the app
yapcode up # first run: setup wizard → starts servers → opens browser

Why brew trust? Newer Homebrew refuses to install from any third-party tap (anything outside homebrew/core) until you trust it once — supply-chain protection, not a warning about this tap specifically. If your brew is older and doesn't have brew trust, skip that line.

Details unique to Homebrew:

• The formula depends_on node, python@3.12, and tmux, copies the source into the Cellar, builds the backend virtualenv and a production frontend build (so the launcher runs next start, with no compile-on-launch), and puts yapcode on your PATH.
• Your config (~/.config/yapcode/.env) and runtime state (~/.local/state/yapcode/) live outside the install and survive brew upgrade and uninstall.

brew upgrade yapcode # later, to update

Using the launcher? yapcode up already handles localhost running — this section is the manual equivalent, useful for development. Network mode applies to everyone.

# Terminal 1 — backend on http://localhost:8000
cd backend && ./run.sh
# Terminal 2 — frontend on http://localhost:3000
cd frontend && npm run dev

On localhost the backend trusts loopback (your own machine: 127.0.0.1, ::1, localhost), so no auth token is needed. The backend never auto-loads VC_AUTH_TOKEN from a .env file — it's opt-in per run mode, so run.sh (and yapcode up) stay zero-config even with a token in your config; it applies only when run-network.sh exports it. Open http://localhost:3000.

Network mode binds 0.0.0.0 over HTTPS/WSS and fails closed without a VC_AUTH_TOKEN.

# Terminal 1 — backend over TLS on https://0.0.0.0:8000 (needs VC_AUTH_TOKEN in backend/.env)
cd backend && ./run-network.sh
# Terminal 2 — frontend over TLS on https://0.0.0.0:3000
cd frontend && npm run dev:network

One-time per device:

1. Generate dev TLS certs at frontend/.certs/dev-key.pem and dev-cert.pem. The cert's SANs must include the device's IP. Copy the template and set your LAN IP under [alt], then:

   cd frontend/.certs
   cp san.cnf.example san.cnf # then edit: set IP.1 to your LAN IP
   openssl req -x509 -newkey rsa:2048 -nodes -days 825 \
    -keyout dev-key.pem -out dev-cert.pem -config san.cnf

   Re-run this whenever your LAN IP changes.

2. Visit https://<host>:8000 once and accept the self-signed cert — otherwise the wss terminal connection is silently blocked.

3. Open the app once as https://<host>:3000/#vc_token=<VC_AUTH_TOKEN>. The browser stores the token in localStorage and strips it from the URL. (The token is never baked into the JS bundle.)

A phone needs HTTPS (a "secure context") for the microphone to work off localhost.

If you ran the first-run wizard, a VC_AUTH_TOKEN was already generated for you — it's in your config file (yapcode config to view). Manual setups can generate one with:

python3 -c "import secrets; print(secrets.token_urlsafe(32))"

The full experience — talk to Claude and watch the live terminal — from your phone, as long as it's on the same Wi-Fi as your computer. The work always happens on your machine; the phone is just a remote.

On your computer (once):

1. Start network mode — both servers over TLS, with the cert's SAN matching your LAN IP.
2. Find your LAN IP: ipconfig getifaddr en0 (macOS) or hostname -I (Linux) — say it's 192.168.1.42.
3. Have your VC_AUTH_TOKEN handy — the wizard already wrote one to your config file (yapcode config to view).

On your phone (once per device):

1. Join the same Wi-Fi as your computer.

2. Open https://192.168.1.42:8000 and accept the self-signed certificate. Skip this and the live terminal is silently blocked (the wss: connection fails without any visible error).

3. Open https://192.168.1.42:3000/#vc_token=<your token> — the app stores the token and strips it from the address bar. Allow microphone access when prompted.

   💡 You don't have to type this URL: npm run dev:network prints the full phone URL (IP + token) at startup — just open it on your phone.

Every time after that: just open https://192.168.1.42:3000 — talk, watch the terminal, approve permission prompts from the couch, the kitchen, anywhere on the network.

If something doesn't work:

• Mic button does nothing → you opened http:// or used a hostname not in the cert — the mic needs a secure context (https://).
• Terminal stays blank → re-visit https://<ip>:8000 and accept the cert again.
• Nothing loads after a router change → your LAN IP changed; regenerate the certs with the new IP (see network mode) and re-accept on the phone.

There is no native Windows support — yapcode drives Claude Code through tmux, which is Unix-only. The supported path is WSL2.

# 1. In PowerShell (Administrator), then reboot and set a Linux username/password:
wsl --install -d Ubuntu-24.04

Use Ubuntu 24.04 — it ships Python 3.12. Ubuntu 22.04 ships 3.10 and 20.04 ships 3.8, both too old (you'll hit the misleading claude-agent-sdk pip error).

# 2. Inside Ubuntu — system deps:
sudo apt update && sudo apt install -y tmux git python3 python3-venv curl
# 3. Node 20+ via nvm:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
exec $SHELL
nvm install 20
# 4. Install Claude Code inside WSL and run `claude` once to sign in.
# 5. Clone into the LINUX home (NOT /mnt/c/...) and start:
git clone https://github.com/nithiink/yapcode.git ~/yapcode
cd ~/yapcode
./bin/yapcode up

Then open http://localhost:3000 in your Windows browser (WSL2 forwards localhost, and the mic only works in that secure context — don't use a LAN IP).

WSL gotchas:

• Clone into the Linux home (e.g. ~/yapcode), not /mnt/c/... — the latter is slow and breaks file watching.
• Set ALLOWED_PROJECT_ROOTS to a Linux path (e.g. /home/<you>/projects) and keep edited projects on the Linux side.

Config lives at backend/.env (chmod 600) — one file, loaded directly by the backend, with no second location and no precedence to track. A Homebrew install instead uses ~/.config/yapcode/.env (its wrapper sets YAPCODE_CONFIG_DIR, since the Cellar is read-only); set YAPCODE_CONFIG_DIR yourself to keep a clone's config out of the tree too. Runtime state (logs, tmux store) lives under <project>/.yapcode/ from a clone, or ~/.local/state/yapcode on Homebrew (override base via XDG_STATE_HOME / VC_SESSION_STORE); the Homebrew config + state survive brew upgrade and uninstall.

The first-run wizard writes this file; yapcode config edits it.

Frontend-side: BACKEND_URL (default http://localhost:8000) is what the Next /api/* proxy forwards to; dev:network sets it to https://localhost:8000 and NODE_TLS_REJECT_UNAUTHORIZED=0.

See backend/.env.example for the full annotated list of every variable.

Under Homebrew, the launcher wrapper exports YAPCODE_ROOT (pins the install tree) and redirects runtime writes out of the read-only Cellar: VC_SESSION_STORE points at ~/.local/state/yapcode/tmux (a subdirectory), while VC_COST_LOG_PATH and VC_DEBUG_LOG_PATH are files directly under ~/.local/state/yapcode.

The two-layer model is summarized up top; this section adds the code-level specifics. The security boundary is the auth token; the Origin allowlist is a convenience for same-network devices, not the boundary.

_access_ok in backend/main.py is the gate. A mismatch returns 401 (when a token is configured) or 403 (when relying on loopback). Token comparison is constant-time (secrets.compare_digest). Even loopback must present a configured token, so the same-origin Next proxy can't launder a remote attacker's request into a "trusted" localhost call. The token is never auto-loaded from a .env file (opt-in per run mode); run-network.sh reads it from the config file and exports it, and refuses to start if none is set.

Token transport. Authorization: Bearer, X-VC-Token header, or ?token= query param (SSE/WebSocket can't set headers). The browser supplies it once via https://<host>:3000/#vc_token=<token> (persisted to localStorage, stripped from the URL). It is never baked into the JS bundle. The token is redacted from access logs (it rides the query string on the SSE debug stream and the terminal WebSocket). The terminal WebSocket authorizes Origin + token before ws.accept(); a disallowed Origin closes with 4403, a bad/missing token with 4401.

ALLOWED_PROJECT_ROOTS confines every session's starting working directory. Every candidate path — including fuzzy-matched ones — is realpath-normalized and containment-checked, so .., symlinks, and absolute-path escapes are blocked. The wizard also rejects roots that are empty, /, $HOME, or non-existent.

Scope note: this bounds where sessions start, not what a running Claude session may touch — actions inside a session are gated by Claude Code's own permission prompts (and act freely in auto mode, as at the keyboard). The voice agent's own tool surface contains no arbitrary-command tool: it can only spawn and drive Claude sessions.

• CORS is restricted to trusted frontend origins (no wildcard), and the Origin allowlist is enforced in-app (not just via CORS response headers) — a disallowed Origin gets 403 even though CORS would hide the response.
• The Next /api/* proxy rejects cross-site requests (via Sec-Fetch-Site, with Origin/Host fallback) to block drive-by CSRF. The frontend dev server binds loopback only (-H 127.0.0.1), so in zero-config localhost mode the proxy isn't reachable from the LAN; LAN access requires npm run dev:network.
• Interactive API docs (/docs, /openapi.json) are disabled so the route/schema map isn't disclosed.

The Origin allowlist is a convenience for same-network devices — the token is the security boundary.

Because each session runs in a tmux session named vc_<id> and tmux allows multiple clients on one pane, you can drive a single Claude session by voice and keyboard at the same time — single process, single transcript, no conflicts.

The bridge is the /voice-handoff command (a skill shipped in the Claude Code plugin at integrations/claude-code-plugin/). It registers the terminal session you're sitting in with your local yapcode backend, so the voice agent can co-drive that exact session. Install the plugin once (user scope, available in every session):

claude plugin marketplace add nithiink/yapcode
claude plugin install yapcode@yapcode

Then, with the backend running, either:

• Start voice-ready (recommended): run the plugin's yapcode launcher (integrations/claude-code-plugin/bin/yapcode — distinct from the top-level yapcode CLI) instead of plain claude. Work normally; type /voice-handoff whenever you want voice — it switches on instantly, no restart. Keep typing there, and open the app to talk.
• From a plain claude session: type /voice-handoff; yapcode reopens the session under voice management and prints a tmux attach -t vc_... command. Press Ctrl-D to leave the old process (single writer per session), then run the attach command to keep typing in the same session while the voice agent drives it too. Only want to drive by voice? Just open the app — no attach needed.

Reaching a remote or tunneled backend instead of localhost: set YAPCODE_URL (default http://localhost:8000) and YAPCODE_TOKEN (required when the backend has VC_AUTH_TOKEN set).

Both clients share one pane, so take turns — don't type and talk in the exact same instant. More detail in the plugin README.

The whole point: a spoken instruction becomes a real tmux action driving the live Claude CLI. The high-level pipeline is in How it works; below is the path through the code.

 You speak
 │
 ▼
┌──────────────────────┐ ephemeral token only ┌───────────────────────────────┐
│ Voice provider │ ◀────── POST /api/session ── │ Frontend (Next.js 16/React19) │
│ OpenAI/Azure (WebRTC)│ │ components/VoiceAgent.tsx │
│ Gemini (WebSocket) │ ── function_call ──────────▶ │ lib/realtime.ts | lib/gemini.ts│
└──────────────────────┘ └───────────────┬───────────────┘
 POST /api/tools/execute (proxy)
 │
 ▼
 ┌──────────────────────────────┐
 │ Backend (FastAPI, main.py) │
 │ POST /tools/execute │
 │ tools.dispatch_tool (tools.py)│
 └───────────────┬───────────────┘
 │ tmux send-keys
 ▼
 ┌──────────────────────────────┐
 │ tmux session vc_<id> │
 │ real `claude` CLI (TUI) │
 └───────────────┬───────────────┘
 tmux capture-pane → narrate back
 PTY over WS → live xterm terminal

(WebRTC and WebSocket are just the realtime audio transports for the two provider families.)

How a voice tool call becomes a tmux action:

1. The voice model emits a function_call (e.g. tell_claude, start_session, answer_prompt). The frontend POSTs it to the Next /api/tools/execute proxy, which forwards to the backend POST /tools/execute.
2. tools.dispatch_tool (backend/tools.py) runs the tool. Long-running ones (tell_claude, run_slash_command, answer_prompt) return {status:"working"} immediately; the frontend polls and narrates when Claude is done.
3. The CLI runner (backend/tmux_runner.py) creates a detached tmux session named vc_<handle[:8]> running the interactive claude CLI. Input is tmux send-keys; output is read with tmux capture-pane. Permission decisions flow through a PreToolUse hook in backend/tmux_hooks/.
4. The browser's live terminal (frontend/components/LiveTerminal.tsx, xterm.js) bridges a PTY-over-WebSocket to the same tmux pane — closing it just detaches; the session keeps running. (This shared pane is what enables co-driving.)

Two Claude execution backends share a ClaudeRunner interface (backend/claude_runner.py; routing between backends lives in backend/session_manager.py): cli → TmuxClaudeRunner (default; real interactive CLI, uses your Max subscription, supports --chrome) and sdk → SDKClaudeRunner (Claude Agent SDK). Each session handle is a uuid owned by one backend.

Unified pipeline event bus (backend/event_log.py): every hop (voice ↔ backend ↔ Claude) is captured into one ordered stream feeding an in-memory ring buffer, live SSE subscribers (the in-app Activity log via /debug/stream), and an append-only debug-log.jsonl. This is your best friend when debugging.

Contributions welcome — this is MIT-licensed and built to be hacked on.

Both servers reload on change, so the inner loop is fast:

cd backend && ./run.sh # uvicorn main:app --port 8000 --reload --reload-dir .
cd frontend && npm run dev # next dev -H 127.0.0.1 -p 3000

• Backend runs with uvicorn --reload. The watcher is scoped to backend/ (--reload-dir .) so the rapidly-written events log doesn't trigger reloads. With detach-on-shutdown, reload preserves running CLI sessions (they rehydrate on restart) — you rarely lose state.
• Frontend runs with npm run dev (Next dev server, loopback-bound on 127.0.0.1).

# Backend deps are lock-pinned with hashes
uv pip sync requirements.lock # install from the lock
uv pip compile requirements.txt -o requirements.lock --generate-hashes # regenerate
uv run --with pip-audit python -m pip_audit -r requirements.lock # audit
# Frontend
npm ci # lockfile-exact install

Tip: the in-app Activity log (SSE from /debug/stream) shows every voice ↔ backend ↔ Claude event live — a great way to understand the pipeline before changing it.

• Docs — clarify setup, expand troubleshooting, improve the WSL2 path, or fix anything in this README that tripped you up.
• Voice providers — both transports implement one VoiceSession interface (frontend/lib/voice.ts: start / stop / injectUpdate / setMuted). Adding or improving a provider is well-isolated work; the backend mint logic lives in backend/main.py (POST /session).
• Integrations — extend the Claude Code plugin in integrations/claude-code-plugin/ (e.g. the /voice-handoff flow).
• New voice tools — add to TOOL_DEFINITIONS + dispatch_tool in backend/tools.py.

• Keep changes scoped; match existing style. The backend pins deps with hashes — if you touch requirements.txt, regenerate requirements.lock.
• Note any new env var, command, port, or flag in the README's configuration reference.
• Test the path you changed locally (the localhost run is zero-config); note which provider(s) / OS you tested on, since transports and tmux behavior vary.
• Be explicit if a change affects the security model (auth, the directory sandbox, CORS/Origin/CSRF handling, redaction) — those need careful review and shouldn't be weakened without discussion.
• Don't commit secrets. The committed cost-log.jsonl / debug-log.jsonl will be scrubbed from history before the repo goes public; don't add to them.

Releases are cut with packaging/release.sh vX.Y.Z (clean tree required; idempotent on the tag). Homebrew distribution goes live only after the repo is public and a tag exists.

MIT — Copyright (c) 2026 Nithiin Kathiresan.