feat(agent): state verification scope on every emitted answer - #3401

🟡 Important

1. A mentioned runner is scored as an executed one (src/gaia/agents/base/verification.py:42, used at :82)

_CHECK_COMMAND_RE.search(command) matches anywhere in the command string, and the shell tool returns return_code == 0 for all of these:

The module docstring commits to the opposite trade ("a false positive would claim a check that never ran"), so this is a gap in the stated contract, not a preference. Anchor the alternation to invocation position and take the runner from group 1:

_CHECK_COMMAND_RE = re.compile(
 # The runner must be INVOKED — start of command or just after a shell
 # separator — not merely mentioned inside it.
 r"(?:\A|[;&|]|\bthen\b|\bdo\b)\s*"
 r"(?:sudo\s+|env\s+\S+=\S+\s+|uv\s+run\s+|poetry\s+run\s+|npx\s+)*"
 r"("
 r"pytest|py\.test|tox|nox"
 # ... unchanged alternation ...
 r")\b",
 re.IGNORECASE,
)

with verification.py:83 reading match.group(1) instead of group(0). Worth a test per row of that table.

2. The scope line is persisted into memory (src/gaia/agents/base/agent.py:6645)

_after_process_query receives result["result"], so MemoryMixin.store_turn writes the line into every stored assistant turn, it is embedded into the FAISS index, and _extract_via_llm sees it as content. It is per-turn process metadata, identical on every turn — not something to remember or retrieve.

 self._after_process_query(
 user_input, strip_verification_scope(result.get("result", ""))
 )

(add strip_verification_scope to the import at agent.py:42)

3. _with_verification_scope is not idempotent (src/gaia/agents/base/agent.py:4476)

Surfaces re-send the answer as history, so from turn two onward every assistant message the model sees ends in Verification: .... Gemma-4-E4B mimics trailing patterns; when it does, the emitted answer carries the model's invented line and the real one, and strip_verification_scope only removes the last. Cheap to make safe:

 if not answer or not answer.strip():
 return answer
 # Idempotent: the model sees this line in history and may echo it.
 answer = strip_verification_scope(answer)
 return f"{answer.rstrip()}\n\n{self.verification_scope_statement()}"

🟢 Minor

• A check that raises is invisible (agent.py:3338, :3350) — _note_verification_signal runs only after _execute_tool returns, so a run_tests tool that raises produces "unverified — no tools ran", which is factually wrong about the run rather than merely cautious. Moving the note into the finally (with failed=True on the raising path) would cover it.
• First command key wins, even when it doesn't match (verification.py:78-83) — the loop returns on the first present key, so {"cmd": "ls", "script": "pytest -q"} reads None. Continue to the next key on a non-match.
• PR description covers only the feature — the diff also lands 7 plan docs, a CI reference, a manual-testing guide, and .gitignore changes. Per CLAUDE.md's PR rules that bundling wants a one-line threads list so a reviewer knows what else is in scope.

Strengths

• The exit-path enumeration is the real work here, and it is right: placing the statement after finalize_answer so a subclass can't drop it, guarding the post-loop catch-all with a flag so nothing is stamped twice, and excluding the console-cancellation path so a cancelled turn stays empty. One test per path, driven through the real loop rather than the helper.
• Deriving pass/fail from _is_error_result means return_code / has_errors from the shell tool are already honoured — a failing pytest reads partially verified, which is the case that mattered most to get right.
• The dependency-free module keeps the SSE handler off agent.py, and the card-echo guard (set the line aside, re-attach only if something survives the cleaners) is a subtle case caught before it shipped.