-
Notifications
You must be signed in to change notification settings - Fork 102
fix(orchestrator): allow wired MCP tools + live agent verification (P2-ii-d) - #1132
Draft
najmuzzaman-mohammad wants to merge 1 commit into
Draft
fix(orchestrator): allow wired MCP tools + live agent verification (P2-ii-d) #1132najmuzzaman-mohammad wants to merge 1 commit into
najmuzzaman-mohammad wants to merge 1 commit into
Conversation
...erification (P2-ii-d) Ran the migration's first live agent turns (Claude Agent SDK driving Claude Code; no key needed — uses Claude Code auth) and caught a production-breaking bug unit tests could not. Bug: ClaudeAgentHarness wired its MCP servers but never ALLOWED their tools. The agent's team_task calls (submit/complete/create/block) appeared in the transcript — so the pure classifier still fired correctly — but were permission-DENIED, so the tool body never ran and the BROKER never saw the side effect. No task would actually be created/submitted/completed; the whole orchestrator path was inert in production while looking fine in classification. Fix: ClaudeAgentHarness._allowed_mcp_tools grants `mcp__<server>` (server-prefix, covers every tool the server exposes) for each wired MCP server, passed as allowed_tools in _build_options. Unit-tested; exercised live. Live verification scripts (real Claude turns, no broker — a stdio team_task MCP stub stands in): - live_harness_check.py — a planning turn classifies plan_ready. - live_decompose_check.py — real team_task create calls -> DECOMPOSED + child specs. - live_service_check.py — full POST /run service path: a submit turn hits the review gate (review/interrupted), a decompose turn creates two subtasks (stub records both). Both PASS with the fix; Turn B was DENIED (0 recorded) before it. Test env-robustness: the SDK-absent degrade tests now skip when claude-agent-sdk is installed (they still run in CI, where it is absent) + a complementary SDK-present test asserts build_harness wires the real harness. pytest green (89 passed, 3 skipped locally). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Important
Review skipped
Draft detected.
Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2ee2eb08-967c-473b-9ea5-9273442b3c9f
You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.
Use the checkbox below for a quick retry:
- 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
- Create PR with unit tests
- Commit unit tests in branch
deepagents/13-live-run
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P2-ii-d — Live agent run + the bug it caught (stacked on #1131)
The first live agent turns of the migration (Claude Agent SDK driving Claude Code — no API key needed, it uses Claude Code's own auth). This both demonstrates the P2-ii gate end-to-end and catches a production-breaking bug that unit tests could not.
🐞 Bug caught by the live run
ClaudeAgentHarnesswired its MCP servers but never allowed their tools. The agent'steam_taskcalls (submit / complete / create / block) appeared in the transcript — so the pure classifier still fired correctly — but were permission-DENIED, so the tool body never ran and the broker never saw the side effect. No task would actually be created/submitted/completed: the orchestrator path was inert in production while looking healthy in classification.Fix:
ClaudeAgentHarness._allowed_mcp_toolsgrantsmcp__<server>(server-prefix — covers every tool the server exposes) for each wired MCP server, passed asallowed_tools. The diagnostic was unambiguous: same prompt,allowed_toolsabsent → 0 tool executions; present → 2.Live verification (real Claude turns, no broker)
A stdio
team_taskMCP stub (scripts/stub_team_task_mcp.py) stands in for teammcp:live_harness_check.py— a planning turn classifiesplan_ready. ✅live_decompose_check.py— realteam_task createcalls flow through the production transcript collection →DECOMPOSED+ extracted child specs (with the dependency). ✅live_service_check.py— fullPOST /runservice path: a submit turn hits the review gate (review/interrupted— the P2-0 gate fix, live over HTTP), a decompose turn creates two subtasks (stub records both). ✅ Both pass with the fix; the decompose turn was DENIED (0 recorded) before it.Tests
SDK-absent degrade tests now skip when
claude-agent-sdkis installed (they still run in CI, where it's absent) + a complementary SDK-present test assertsbuild_harnesswires the real harness.pytestgreen (89 passed, 3 skipped locally).Grade
See the PR discussion / session summary — the gate (decompose → coordinate → dispatch → completion) is proven in mechanics (unit + Go E2E) and the live seams (inner harness, classification, service path) are now demonstrated against a real agent. The remaining step is a full single-process broker→sidecar→agent boot (heavier integration), deferred.
🤖 Generated with Claude Code