fix(eval): capture stdout when a scenario subprocess fails - #3375

Verdict: Approve

A failing eval scenario now tells you why it failed. The runner only read stderr, but the Claude CLI in JSON mode writes its own error to stdout — so every failed scenario recorded an empty error and CI failures had to be reproduced locally to triage. This reports both streams, and names the exit code when a process dies silently.

The change is small, the three new tests pin each branch (stdout-only, both streams, and the never-empty fallback), and the before/after evidence shows the real fix landing. Nothing blocking.

One thing worth a follow-up, not a merge blocker: a scenario that times out still records no explanation at all, which is the same triage gap this PR closes for the crash path.

Real-world evidence

The PR ships an evidence bundle, and the fix is exercised through the real gaia eval agent command against a live backend on 127.0.0.1:4200 (/api/health → HTTP 200). Same command run at HEAD~1 and HEAD:

Before (HEAD~1):
[ERROR] concise_response — exit code 1
 ← stderr line printed empty
{ "status": "ERRORED", "error": "" }

After (HEAD):
[ERROR] concise_response — exit code 1
stdout: {"type":"result","subtype":"error_during_execution","is_error":true,"result":"Not logged in - please run /login"}

All four branches were exercised live — stdout-only/exit 1, both streams/exit 2, silence/exit 3 (no output on either stream (exit 3)), and the untouched success path (PASS 9.0/10, 1/1 passed (100%)). The JUnit consumer of the changed field was checked too, and now carries the captured text instead of a blank <error> body. Spot regression poked --compare, --audit-only, and three sibling backend routes — all clean. pytest tests/test_eval.py::TestRunScenarioSubprocess -q → 20 passed.

The bundle is explicit that no real LLM turn ran (no Lemonade/GPU on this runner) and that a scorecard-vs-baseline comparison is pending strix-halo lane, and that the failing claude subprocess was a PATH stand-in — everything downstream of it (the CLI, runner, backend, trace files) was real. That's the right shape for this change: the code under test is the failure-handling path, which the stub reaches faithfully. Agent UI screenshot is correctly N/A — this writes a field consumed by CI, not the UI. The verdict rests on this evidence plus static review; nothing here contradicts the fix.

 except subprocess.TimeoutExpired:
 elapsed = time.time() - start
 print(f"[TIMEOUT] {scenario_id} — exceeded {timeout}s", file=sys.stderr)
 result = {
 "scenario_id": scenario_id,
 "status": "TIMEOUT",
 "overall_score": None,
 "turns": [],
 "error": f"subprocess exceeded {timeout}s timeout",
 "elapsed_s": elapsed,
 "cost_estimate": {"turns": 0, "estimated_usd": 0.0},
 }