Milestones
List view
Close the gap between what GAIA's agent loop can do and what running a real development task requires. Substrate work, not model work. ORDER OF ATTACK (dependency edges matter): Wave 1 - independent, highest impact: #3373+#3374 shell bypass (PR #3394) | #3376 claim downgrade | #3377 ambiguous-edit bug Wave 2 - after the shell PR merges (same files): #3380 persistent shell | #3382 adaptive timeouts Wave 3 - depends on #3376 for the exit-path enumeration and on #3373/#3374 for a runnable check: #3381 verification state Independent, lower impact: #3379 project map | #3378 addressable output | #3384 typed git | #3385 read cache | #3386 steering | #3383 delegation ADOPTED, previously unmilestoned: #2768 read-only tier sweep | #2785 security-model doc drift (note: the invariant it asks for no longer exists in code - see comment) | #3365 Agent UI CSRF/CORS, which is a hard dependency for #3373 (bypass must never reach the HTTP transport). Two constraints for anyone picking these up: evals must run SERIALLY - the local backend is single-tenant per model slot; and several issues carry C++ prior art from #2810 that should be ported rather than reinvented.
No due date•0/22 issues closedTriage and burn-down of the 66 open GitHub code-scanning alerts (CodeQL + Semgrep OSS). Every alert was verified against the real code: 16 are genuine and split across 6 fix issues; 50 are false positives from 4 rules and are dismissed here, with the scanner config tuned so they do not refill.
No due date•2/7 issues closedRESCOPED. The old title promised parity with domain agents that no longer exist (#2995) — that premise is void. The contents are not: all of this is skill-runtime, MCP, RAG and TUI capability for a native binary, which is exactly the post-collapse direction. The keystone is a generic skill-programmable gaia-agent target. --- (Previous scope, retained for history) Bring the C++ framework to parity with the Python SDK on the subsystems domain-specific agents need: retrieval (embeddings + vector index), structured persistence, SKILL.md runtime, and a complete MCP client — plus native OpenAI tool calling. Reference deliverable: gaia-code, a native C++ coding agent. Plan: docs/plans/cpp-framework-parity.md
No due date•21/35 issues closedSkills that learn. An immutable authored base (milestone 60) plus a per-agent learned overlay held in memory, so an agent adapts a generic skill to its actual user without ever mutating the shared file. Covers the overlay store, the typed delta grammar, rebase-on-base-update, the injection-resistant write path, off-states, and the graduation bridge back to authored skills. NOT started until milestone 60 ships and is validated in the real world.
No due date•0/10 issues closedAggregates all work on GAIA Agent Skills — the agentskills.io-based SKILL.md format, runtime loader (gaia.skills), per-agent scoping, permission/security-tier model, marketplace, and Claude Code / Hermes / OpenClaw compatibility. Umbrella feature: #1019. Broader issues that only partially touch skills (OEM bundling #648, config dashboard #701, self-improving agent #553, Agent Factory #1111/#1121) are intentionally kept in their own milestones.
No due date•13/21 issues closedRESCOPED. Dropped the (2026-07) date — it was the only stale thing about this milestone. An audit checked all 53 open issues against the tree: 47 still reproduce and several have open PRs. Note that roughly two thirds were never email issues at all — they are TUI, daemon, connector, core-agent and CI bugs found during triage runs, and are being moved to their real homes. --- (Previous scope, retained for history) Email-agent validation and fixes. PHASE 1 — COMPLETE: consolidated bugs from the full T4–T25 + autonomy A0–A6 regression run (2026年07月23日, Ryzen AI iGPU laptop, source build) driven through the Agent UI. All fixes merged, incl. batch id-coercion (#2455), cross-request bulk undo (#2456) and the batch-tool retry dead-end (#2464). Historical index: #2431. PHASE 2 — COMPLETE: full 58-check capability sweep through the new 'gaia tui' terminal UI. Produced #2620–#2627, #2743–#2745, #2756, #2759. Historical index: #2511. PHASE 3 — CURRENT: scope narrows from capability breadth to answer quality. The email agent's job is making sense of a mailbox, not managing one — four sense-making jobs held to a high bar, each tested against multiple unrehearsed phrasings so no prompt path is ever hardcoded. Start here: #2762 (Email Agent Validation Index).
No due date•182/230 issues closedRESCOPED. The port never happened — #2995 deleted eleven of the thirteen agents this was going to port. What survives is the quality bar the port work uncovered, applied to the three real packages: manifest truth, generated capability matrix, the four-file doc bundle, a scorecard gate, and a working release lane. --- (Previous scope, retained for history) Take the 17 unpublished in-repo agents (plus the in-core BuilderAgent) to a shippable bar — and ship only the ones that should exist. This is not a lift-and-shift: several agents do not do what their manifest advertises, and none is validated well enough to publish as-is. TRIAGE RESULT (#2315, 13 of 18 decided): roughly 5-6 become published catalog agents. PORT: docqa (sole RAG survivor), summarize, docker, sd (as a shared generation layer). MERGE: doc-search -> docqa; emr -> A12 health (#1496); jira -> L8 TaskStore. DISCARD from catalog: blender, hello-world, word-count, routing. Infrastructure: connectors-demo (defer), builder (stay in-core). Verdicts still open: chat, fileio, browser, analyst, code. Porting all 18 blindly would have been ~500 files of parity kit for packages that duplicate #48's more general agents or are not products at all. State of play (audit at HEAD, post-#2060 agent-first layout): 1 of 18 agents is actually published — email (npm @amd-gaia/agent-email 0.5.0; the live hub index.json lists only 'email'; zero agent wheels on PyPI). The other 17 ship a gaia-agent.yaml, a README, and a smoke test. None has an eval scorecard. src/gaia/daemon/sidecars/spec.py registers exactly one AgentSidecarSpec, and it is email. All audited agents declare 'api_server: true' with no REST surface behind it — publishing as-is ships a false capability claim. ORDERING PRINCIPLE — generalize before documenting. Every port runs: (1) capability-truth audit — does the agent actually do what its manifest claims, and is it general or demo-shaped; (2) generalize + harden to close that gap; (3) THEN the surface, docs, scorecard, and release. Documenting first means SPEC/SKILL/SCORECARD get written against behavior that is about to change. Track A builds the per-agent port kit once (the surface the Agent Factory automates). Track B applies it, per verdict. Track C closes quality gates that block publishing regardless. Audits: docs/plans/port-audit-6-agents.md, docs/plans/port-triage-thin-infra-agents.md, docs/plans/port-triage-external-dep-agents.md. Depends on: #42 (Agent Factory), #51 (registry-driven publisher). Distinct from #48, which builds 23 net-new agents.
No due date•30/50 issues closedAgents distributed as independent packages, each with its own tools and permissions, that delegate tasks to each other and pass content between them. Chat cannot touch the filesystem — it requests work from the file agent. MVP: chat, doc, file; architecture scales to arbitrary N. Every agent stands on its own. Plan: docs/plans/agent-mesh-milestone.md
No due date•4/8 issues closedFully-autonomous, always-on agents on local inference (Lemonade/NPU) for ambient OPS work — GitHub inbox, task hygiene, workspace management, fleet observation — while Claude Code remains the coding agent. Claudia (used 8-10h/day) is the dogfooding testbed. VALUE RANKING (post-scrutiny): 1 #2210 lifecycle (half-landed; finish auto-respawn+budgets) · 2 #2211 wire envelope · 3 #2214 decision traces · 4 #2213 run registry · 5 #2256 shared code index (strongest evidence; fixes the 4B/16K envelope) · 6 #2255 permission broker (largest quantified pain; phase-2 behind #1465) · 7 #2217 Daily PR Audit (pipeline proof) · 8 #2221 Fleet Observer. PHASE 1 (contradiction fixed): #2210, #2211, #2213, #2214, #2217, + #2256's first slice (branch/worktree-aware freshness as SDK extension — no claudia coupling). #2220 is FIRST AFTER the frozen claudia API contract exists, not phase-1. Phase 2+: #2212 (behind #1465), #2215 (narrowed), #2216 (epic over #43), #2255, mutation + detection agents. Six issues closed after scrutiny (#2218/#2219/#2224/#2228/#2229/#2233 — scope absorbed by survivors). Comments on repaired issues are authoritative where bodies lag.
No due date•8/26 issues closedTransition to the v2 architecture (#1913): every agent runs as an out-of-process sidecar under a headless custodian daemon; the Agent UI becomes a thin client synced to the Agent Hub — browse, install/uninstall, and run agents (npm packages + frozen sidecar binaries) directly from the UI. Owns: daemon + /host/v1/* custody, AgentSidecarManager supervision, the /query SSE contract + render→component map, auth legs, model-slot broker, scheduler-clock relocation, AND the full hub distribution loop absorbed from milestone 52 (multi-component catalog/publish, in-app install/run, signing). Plans: docs/plans/agent-ui.mdx (v2 section), agent-ui-agent-capabilities-plan.md §0, agent-ui-v2-implementation-breakdown.md. Boundary: the autonomy policy layer stays in milestone 6.
No due date•63/65 issues closedCut MCP prompt + result token cost without code execution. Route MCP tools through the dynamic tool loader (progressive disclosure), keep large tool results out of model context via a session-scoped artifact store, and (optionally) add PII pass-through for connector-to-connector flows. Plan: docs/plans/mcp-context-efficiency.md. Non-goal: model-authored code execution / sandbox.
No due date•1/5 issues closedMeasure and surface agent adoption across the hub. Zero-telemetry stays the DEFAULT (#890); usage analytics are opt-in, asked ONCE at first install, with a plain-language disclosure of exactly what is tracked (#890 section-F opt-in path). Sources: R2 download counts + unique installers + trend + version adoption at the agent-hub Cloudflare Worker → Analytics Engine (#1819); opt-in runs + install outcomes (#1820); external-registry counts across npm/PyPI/GitHub releases (#1754); and an auth-gated internal admin dashboard via Cloudflare Access SSO that aggregates them all (#1821).
No due date•0/4 issues closedExtend the Agent Hub from agents-only to a multi-component registry (app · component · agent), publish the Agent UI as a first-class `app` (R2 + npm, mirroring the email agent), and add an in-app marketplace that installs Hub units on demand. Scope + gap inventory: docs/plans/agent-ui-hub-publish.mdx (PR #1714). P0 = MVP must-haves (publish→browse→install loop + multi-component plumbing); P1/P2 = UX, speed, and polish.
No due date•2/13 issues closedOne registry-driven, idempotent publisher for every shippable GAIA artifact — hub agents (npm + binary + wheel), the Agent UI npm package, and future packages (memory). Replaces the one-off release_agent_email.yml with a scalable model supporting selective, coordinated, and release-everything-skip-unchanged releases. Plan: docs/plans/package-publishing.mdx. Follows v0.23.0 (#49), which ships the first real email release the manual way.
No due date•0/8 issues closedRESCOPED. v0.23.0 shipped, so the release-train framing is spent — but none of these issues shipped in it. They are real, unstarted packaging and eval work: signing, arm64 binaries, eval corpus parity, release gates, and the paused publish lane. --- (Previous scope, retained for history) **Email Agent Packaging — native C++ + npm distribution.** Takes the frozen, eval-validated Python email agent (Phase 1, v0.21.0 / #40) and delivers the production distribution: a thin JS/TS npm client plus per-platform self-contained binaries (frozen Python + native C++) run as a loopback-REST sidecar behind one OpenAPI contract. Plan: docs/plans/email-agent-packaging.mdx. **Scope** - OpenAPI REST contract + /version (#1645); JS/TS npm client (#1646). - Frozen Python per-platform binaries (#1647); npm optionalDependencies packages + atomic publish (#1648). - Native C++ build: transpile (#1118), launcher / per-agent reimplementation (#1110, #1517), cold-runnable signed artifact + parity & contract-conformance gate (#1543). - Signing/notarization: Windows code signing via SignPath (#732), macOS Apple Dev ID notarization (#733), sign the distributed binaries (#1650). - Namespaced agent-pkg-* tags; paused PyPI path (#1179) untouched. **Depends on:** v0.21.0 (#40) — frozen request/response contract + eval baseline + Python agent.
No due date•26/43 issues closedRESCOPED. The 23-agent catalog is gone (#2995) — GAIA ships one flagship plus chat and email, and a capability is now a hub/skills/<name>/SKILL.md, not a package. What this milestone tracks is the shared infrastructure those agents were going to sit on: connectors, VLM/media, notifications, the cross-agent task store, and the remaining personalization gaps. --- (Previous scope, retained for history) Ship 23 purpose-built, proactive agents as Hub-native packages over 7 shared infrastructure layers. Each agent is one compiled package (TUI/CLI/pipe/API/MCP) published to R2 and surfaced on the website + Agent UI, autonomous-by-default with local personalization. Spec: docs/plans/agent-hub-22-agents-spec.md. Depends on Agent Hub Platform (v0.29).
No due date•21/85 issues closedRESCOPED. The Strix Halo half shipped — seven jobs now target the self-hosted pool and #3262 routed them to the Ryzen Dev Lab. What remains is the installer half: actually installing and launching the built artifacts, and a real Playwright suite (tests/electron/ is Jest with mocks today). --- (Previous scope, retained for history) Carry-over from v0.20 — Test & CI Hardening [OSS] (milestone #29). Focuses the remaining test/CI/infrastructure hardening work on real AMD hardware (Strix Halo self-hosted runners) and the installer release matrix, plus two adjacent feature tracks carried over (FLM/NPU agent, Agent UI visual uplift). ## What ships - Strix Halo self-hosted runner for CI workflows (#1297) and the Playwright + Strix Halo E2E suite for the Agent UI (#883) - Installer release integration tests: scenario matrix (OS x UV x Lemonade), self-hosted runner infra, pre-release artifact handling (#989 #990 #991 #992) - Closing out the deep CI/CD + test-coverage audit (#875) - Carried-over features: native FLM format + Ryzen AI NPU agent/profile (#1220), Agent UI visual design uplift via Claude Design (#1001) ## Issues CI / real-hardware: #1297 #883 #875 Installer matrix: #989 #990 #991 #992 Carried-over features: #1220 #1001
Overdue by 1 month(s)•Due by July 14, 2026•4/14 issues closedRESCOPED. 'Agent Hub' is dead — #3120 retired the TUI hub browser and boots straight into the flagship agent, so the browser, detail view and agent-lifecycle screens are gone by design. Six issues here already shipped. What remains is genuine TUI work: rich-rendering parity, connector onboarding UX, real end-to-end coverage, and the daemon convergence tracker. --- (Previous scope, retained for history) Go/Charm-based TUI that serves as the terminal-native Agent Hub. Browse, download, run, and validate agents from one interface. Installed once; downloads and runs any agent on demand (agents do not bundle the TUI). Replaces the Rich readline loop as the default gaia terminal experience. Equivalent to Agent UI + Agent Hub but for the terminal. ~5-8MB static binary, zero runtime deps.
No due date•8/21 issues closedRESCOPED. The knowledge agent's behaviour already ships as the research-report, document-brief, source-watch and rss-digest skills. What is still broken and blocks every web-facing skill is that BrowserToolsMixin.search_web() is hardcoded to DuckDuckGo scraping and never routes to the configured Tavily connector. Plus the one real gap: site-crawl into a local KB. --- (Previous scope, retained for history) Web intelligence + local knowledge fusion agent. Builds persistent, queryable knowledge bases from websites and local documents, then reasons across them with a local LLM. ## What Ships Standalone `gaia-knowledge` CLI + MCP server: crawl-to-RAG pipeline, hybrid web+local research, topic monitoring, Tavily-powered search/extract. ## Core Differentiator The only tool that builds persistent local knowledge bases from the web, fuses them with private documents, and reasons across everything with a local LLM. What ChatGPT/Perplexity can't do. ## Phases - **Phase 1:** Crawl → Index → Query (turn any site into a local KB) - **Phase 2:** Hybrid Research (TavilyHybridClient + local fusion) - **Phase 3:** Topic Monitoring (ambient web intelligence) - **Phase 4:** Polish + Integration (backend abstraction, ecosystem upgrade) Plan: `docs/plans/knowledge-agent.mdx`
No due date•3/8 issues closedRESCOPED. There is no GitHub agent — the capability shipped as hub/skills/github-triage/SKILL.md with its own gh grant and a three-tier write policy. What remains is PR-side work that skill does not do: an impact-ranked review queue, re-review briefings, release prep, contributor SLA tracking, and upstream-bump analysis — each a new or extended SKILL.md. --- (Previous scope, retained for history) Ambient project intelligence + event-driven automation agent for GitHub repositories. One agent, many surfaces (Agent UI / TUI / CI workflow.yml / service / CLI) covering PR review, PR/issue triage, issue review, conversation, auto-fix, release engineering, postmortems, and proactive audit — consolidating the maintainer pull-tooling (#1123–1137) with the event-driven-bot half. Plan: `docs/plans/github-agent.mdx` MVP: issue_triage (dry-run). Phasing in the plan §11; full issue map (existing + new G1–G12) in §13.3.
No due date•6/27 issues closedRESCOPED. The factory's pipeline premise survives; its output does not. Generating a Python agent package from a spec is dead (#2995), but the loop around it — read real usage, author a capability, score it, iterate — is what the skills era needs. Stage one merged as #3111. Every stage is rescoped from 'produce an agent' to 'produce and improve a SKILL.md'. --- (Previous scope, retained for history) Agent Factory pipeline: automated production of specialized, optimized, natively compiled agents from SKILL.md specifications. Takes a SKILL.md spec + target hardware profile and produces a tested, optimized agent via: Hermes orchestrator → Claude Code scaffold → eval sandbox → optimize loop → (optional) model distillation → C++ transpilation → hardware-aware packaging. Proving ground: email triage agent end-to-end. Plan: docs/plans/agent-factory.mdx Depends on: v0.29 — Agent Hub Platform (milestone #21)
No due date•1/10 issues closedRESCOPED. Two issues referenced CLI commands setup.py does not ship and are closed. The rest is documentation rot that is getting worse: this milestone owns the guide-prerequisites problem (#1074, #1077), which the nightly audit independently rediscovered fourteen times in one night because it could not see an existing milestoned issue. --- (Previous scope, retained for history) Patch release focused on first-run reliability, documentation accuracy, and Agent UI stability. Targets gaps surfaced by a full install-to-usage sweep on a clean Windows developer environment. ## What Ships - **First-run experience** — developer quickstart extras (`[rag]`, `[ui]`), VLM profile model registration, prerequisites blocks across 14 guides - **CLI surface audit** — docs reconciled against actual CLI surface (`gaia code`, `gaia mcp add`, `agent.yaml`); `gaia-code index` argparse hardening and `--base-url` help-text drift fixed - **Documentation sweep** — stale Lemonade port (`:8000` → `:13305`), version references, wrong on-disk install paths, broken `#agent-ui-fastest` anchor, FAQ model list - **Connector & API bugs** — `list_documents` reindex endpoint + `last_error` field, email connector OAuth CLI gap, undocumented `connectors` and `telegram` CLI commands, connector pages linked to upstream provider docs - **Agent UI stability** — static-mount caching on startup, CUSTOM AGENTS settings panel inconsistency with CLI, MCP bridge session routing and error-envelope normalization, `gaia docker` security prompt rendering
Overdue by 1 month(s)•Due by July 28, 2026•11/21 issues closedConsolidate all standalone apps (Jira, Docker, LLM, Summarize, Blender) into the main Agent UI. Epic: #759
No due date•14/18 issues closedRESCOPED. The orchestration issues died with the agent collapse. What is left is the eval-to-training flywheel: eval-to-training pipeline, QLoRA/GRPO on AMD GPUs, adapter hot-swap, and model quality optimization. Installer and distribution issues are being moved out — they never belonged with fine-tuning. --- (Previous scope, retained for history) Close the eval→training loop. Make small local models dramatically more reliable on GAIA agent tasks. ## What Ships Eval-to-training data pipeline, Unsloth/QLoRA training on AMD GPUs, LoRA adapter hot-swap, sub-4B model evaluation + first SFT fine-tune. ## Use Cases Enabled - **Reliable tool calling** — agents that consistently pick the right tool with correct parameters - **Smaller, faster models** — sub-4B models fine-tuned to match larger model quality on specific tasks - **Continuous improvement** — eval failures automatically become training data for next iteration - **Domain-specific agents** — per-vertical LoRA adapters (email, code, finance) ## Value Proposition *"Small models that punch above their weight — fine-tuned on YOUR tasks, trained on YOUR hardware, running at NPU speed."* ## The Quality Flywheel ``` Eval (v0.18.0) → training data (#666) → GRPO fine-tune (#667) → LoRA adapter (#668) → deploy via Lemonade → re-eval → repeat ``` ## Key Technical Context - System prompt: 1,477 lines (~2-3K tokens) — must compress for sub-4B models - Tool registry: 35+ tools — reduce to 10-15 essential for small models - Min viable context: 8K tokens (rules out 4K-only models) - Tool format: custom JSON (not OpenAI function calling) - Research shows: 350M fine-tuned model achieved 77% tool calling vs 26% for GPT-class on specific tasks ## Parallelization Plan (multiple devs + coding agents) - **Dev 1:** #666 → #667 (data pipeline → training infra) - **Dev 2:** #616 (prompt compression + model evaluation matrix) - **Dev 3:** #668 (LoRA adapter deployment once training produces output) ## Key Deliverables - Eval-to-training data extraction (#666) - Unsloth QLoRA + GRPO on AMD consumer GPUs (#667) - LoRA adapter library with Lemonade hot-swap (#668) - System prompt compression + sub-4B model eval + first SFT (#616)
No due date•4/12 issues closedAgent selection UI, API readiness for third-party integration, and web app workflow automation. ## What Ships Agent registry with vertical templates, OpenAI-compatible API polish, execution cancellation, web app workflow recorder. ## Use Cases Enabled - **Vertical agent marketplace** — select Email Agent, Finance Agent, DevOps Agent from registry - **Web app workflows** — fill expense reports, extract CRM data, automate SaaS tools via Playwright (#661) - **Third-party API integration** — any app can call GAIA agents via OpenAI-compatible API - **Financial tracking** — browser-based bank dashboard monitoring, spending categorization ## Value Proposition *"Pick the right agent for your job — email, code, finance, DevOps. One click to switch. Any app can integrate via API."* ## Key Deliverables - Agent registry with selection UI - Web app workflow automation (#661) - API fixes for third-party integration - Execution cancellation and context management - UI polish and stability
No due date•33/57 issues closedC++ agent framework for OEM integration — enabling hardware manufacturers to ship pre-installed AI agents on Ryzen AI PCs. ## What Ships Native C++ agent runtime, OEM bundling framework, zero-touch deployment. ## Use Cases Enabled - **Pre-installed AI agent on new PCs** — buy a Ryzen AI laptop, agent works out of the box - **OEM-customized agents** — Acer, ASUS, Dell, HP, Lenovo ship hardware-appropriate defaults - **Agent Computer (Halo Box)** — flagship hardware + software experience - **Embedded agent experiences** — lightweight native runtime for resource-constrained devices ## Value Proposition *"Buy a new AMD PC, your AI agent works out of the box — no setup, no downloads, no configuration. The Agent Computer experience."* ## Key Deliverables - C++ agent runtime optimized for Ryzen AI - OEM pre-configuration framework (default models, skills, settings per hardware SKU) - Zero-touch deployment on new devices - End-to-end integration testing in WebUI (#529)
No due date•14/30 issues closed> **Deferred from v0.18.0 → v0.18.3.** Originally targeted v0.18.0; the v0.18.0 release shipped Connectors framework, ChatAgent expansion, and Memory v2 instead. Agent Eval Benchmark work continues here under v0.18.3. > > **Renamed 2026年05月14日 from "v0.18.0 — Agent Eval Benchmark".** See milestone v0.18.0 (#38) for what shipped under that version label. Agentic eval framework that validates GAIA agent quality using Claude Code as user simulator + judge. The foundation for the eval→fine-tuning quality flywheel. ## What Ships Automated eval harness, ground truth test suite, quality metrics dashboard. ## Use Cases Enabled - **Validated agent reliability** — every release tested against real workflows before shipping - **Quality regression detection** — catch tool-calling failures, hallucinations, format errors - **Training data generation** — eval results feed directly into v0.19.0 fine-tuning pipeline ## Value Proposition *"Agents you can trust — every release is tested against real workflows. When something breaks, it's caught before it reaches you."* ## The Quality Flywheel ``` Eval runs → identifies failures → failures become training data (v0.19.0) → GRPO fine-tuning improves model → re-eval confirms improvement → repeat ``` ## Key Deliverables - Claude Code-based eval harness (user simulator + judge) - Ground truth test suite for core agent capabilities - Pass/fail metrics with tool trace analysis - See docs/plans/agent-ui-eval-benchmark.md for full spec
No due date•16/22 issues closedRESCOPED. Written against Lemonade v9.4.1/v10 while the repo now pins 11.8.1 — version numbers and endpoint paths need re-verification before any issue here is picked up. Issues specifying the deleted SD and Voice agents are closed; the endpoint work (image edit/variations, ASR, TTS, reranking) survives as tool mixins on the flagship. Desktop control / CUA is a separate program and is being split out. --- (Previous scope, retained for history) Give GAIA eyes, ears, and voice via Lemonade v10. Speech (ASR/TTS), vision (image understanding, generation, editing), desktop control (CUA), and browser automation for rich multimodal agent experiences. ## What Ships Lemonade v10 multimodal integration (image gen/edit, ASR, TTS), desktop control via CUA, browser automation upgrades, multimodal rendering in Agent UI. ## Use Cases Enabled - **Home automation via voice** — "Turn off the lights" spoken naturally, processed locally via Whisper ASR - **Healthcare image processing** — medical document scanning, image analysis via VLM - **Receipt scanning for finance** — take photo → VLM extracts items/totals → budget categorization (#664) - **Desktop workflow automation** — agent sees your screen, clicks buttons, fills forms (CUA) - **Voice memos and transcription** — speak thoughts, agent transcribes and organizes - **Creative content** — image generation, editing, illustrated stories via Stable Diffusion ## Value Proposition *"Talk to your agent, show it images, let it read your screen and fill out forms — eyes, ears, and voice, all processed on-device via Lemonade v10. No cloud, no latency, no privacy concerns."* ## Key Deliverables - ImageToolsMixin — text-to-image, edit, variations via Lemonade v10 (#539) - VoiceToolsMixin — ASR transcription + TTS speech (#539) - DesktopToolsMixin — screenshot + mouse/keyboard CUA (#460) - Workflow learning and replay via Playwright (#544) - Financial tracking agent with receipt scanning (#664) - Multimodal rendering in Agent UI (#540) - Lemonade v10 installer and version gating (#533)
No due date•13/45 issues closedWindows OS intelligence via MCP — 9 domain MCP servers for deep system management, turning GAIA into a full PC management agent. ## What Ships Native C++/.NET MCP servers for process, network, storage, thermal, security, startup, gaming, productivity, and backup management. ## Use Cases Enabled - **PC health management** — "Why is my computer slow?" → agent diagnoses and fixes - **Security monitoring** — malware detection, firewall management, update enforcement - **Gaming optimization** — auto-tune settings per game, manage GPU/CPU profiles - **Backup automation** — scheduled backups with intelligent file selection - **Startup optimization** — identify and disable bloatware, speed up boot time - **Network diagnostics** — "Why is my WiFi slow?" → agent runs diagnostics and suggests fixes - **Storage management** — identify large/unused files, optimize disk usage ## Value Proposition *"Your agent manages your entire PC — updates, security, performance, backups, gaming settings. Like having IT support that never sleeps, running entirely on your machine."* ## Key Deliverables - 9 domain MCP servers (C++/.NET): process, network, storage, thermal, security, startup, gaming, productivity, backup - Depends on v0.22.5 (C++ Framework Production Readiness) - See docs/spec/os-agents-mcp-milestone.md
No due date•2/25 issues closedRESCOPED. The memory half shipped — context scoping, temporal awareness, the dashboard and its API, and conversational bootstrap are all done. What remains is voice, dashboards and onboarding UX. Dropped 'Skills' from the name; that work lives in the Agent Skills milestones now. --- (Previous scope, retained for history) Three-tier memory system, conversational onboarding, and deep personalization — the killer feature for local agents. ## What Ships Session memory, long-term memory (MEMORY.md), daily activity logs, system scanner, conversational bootstrap, personalized daily briefs. ## Use Cases Enabled - **Personalized daily briefs** — morning digest of email + calendar + news, tailored to YOUR interests (#663) - **Email habit learning** — agent learns which senders matter, how you categorize, your response style - **Writing style adaptation** — drafts that sound like YOU, not generic AI - **Personal knowledge capture** — "second brain" that remembers everything you've discussed - **Schedule preference learning** — agent knows you prefer morning meetings and blocks afternoon focus time ## Value Proposition *"Your agent learns how you work and gets smarter every day. Your email habits, writing style, schedule preferences — a deeply personal profile that never leaves your device. The longer you use it, the more valuable it becomes."* ## Why Personalization Is the Killer Feature Cloud agents can't store your behavioral patterns without transmitting them to third-party servers (the exact vulnerability behind 42,900 exposed OpenClaw instances). Local memory means personalization stays on-device — human-readable, auditable, editable, and private. This creates hardware retention that cloud subscriptions cannot match. ## Key Deliverables - Three-tier memory: session state, long-term context, daily logs - Local vector embeddings for semantic retrieval - Conversational onboarding ("What do you do? How can I help?") - System scanner (hardware, installed apps, workflow detection) - Personalized daily briefs (#663) - Memory Dashboard UI (#575)
No due date•27/47 issues closedRESCOPED. The recurring scheduler shipped (gaia schedule, the daemon scheduler, the UI panel). Telegram shipped. The deliverables written as agents — calendar, finance, CRM, home automation, infra monitoring — are skills now, not agents. What remains is messaging hardening (security, rate limiting, Discord/Teams/Signal) and the scheduler gaps. --- (Previous scope, retained for history) Always-on autonomous agents that work in the background — scheduling, monitoring, and acting without prompting. The infrastructure that makes agents truly useful. ## What Ships Autonomy engine (heartbeat, scheduler, event hooks), messaging adapters (Telegram, Discord, Slack), workflow orchestration. ## Use Cases Enabled - **Overnight email triage** — agent clears 4,000+ emails while you sleep, morning digest ready (#645) - **Infrastructure monitoring + self-healing** — scheduled health checks, auto-restart, alerts (#665) - **Calendar management** — proactive daily agenda, conflict detection, meeting prep (#662) - **Home automation** — scheduled routines, sensor monitoring, energy optimization (#646) - **Messaging-native interaction** — text your agent on Telegram, get responses (#635) - **Overnight code generation** — agent writes code, runs tests, reports results by morning ## Value Proposition *"Your agent works while you sleep — triages email, monitors servers, manages your calendar, generates code. Wake up to completed work. Zero cloud cost, zero data exposure."* ## Why This Changes Everything This is the shift from "assistant you ask" to "agent that acts." Combined with memory (v0.20.0) and browser automation (v0.18.1), the agent has everything it needs to work autonomously on real tasks. The always-on cost on local hardware: 0ドル. On cloud APIs: 600ドル-3,600ドル/year. ## Key Deliverables - Autonomy engine: heartbeat, cron scheduler, event hooks - Messaging adapters: Telegram, Discord, Slack (#635) - Email Triage Agent (#645) - Calendar Agent (#662) - Infrastructure monitoring agent (#665) - Home Assistant integration (#646) - Agent UI MCP Server for programmatic UI control (#549)
No due date•15/37 issues closed- No due date•27/62 issues closed