-
Notifications
You must be signed in to change notification settings - Fork 162
feat(agent): materialize a project map at task start - #3405
feat(agent): materialize a project map at task start #3405kovtcharov-amd wants to merge 2 commits into
Conversation
The flagship opened every task blind — it guessed directory names, guessed which commands existed, and guessed which shell it was talking to, and each wrong guess cost a full round trip to learn something one orientation pass establishes once. It now gets a 600-token block in the system prompt naming the project root, the directory shape two levels deep, the likely entry points, which commands are installed (and which are not), the three platform differences that change command syntax, and whether the semantic code index is built. The binary half extends the existing day-0 system-context probe rather than adding a second one: `probe_binaries` is now the single PATH probe, shared by `collect_system_info` and the map, and `DEV_TOOL_PROBES` widens the seven desktop-app markers to the developer toolchain. `index_codebase` gains its first automatic trigger. When the root satisfies `is_code_repository` — a VCS directory or a recognised manifest, non-recursive — and no index exists, the map starts one in a background thread and says so in the prompt. `GAIA_PROJECT_MAP_AUTO_INDEX=0` turns it off. Budget is 600 tokens, 1.8% of the 32K NPU window, enforced by `render_project_map` on every render and pinned by a test against a 200-directory repository. The map is cached per root and invalidated by a fingerprint over the top-level listing, manifest contents, VCS head and PATH. (cherry picked from commit 23573da)
... itself Review follow-ups on the task-start project map, each a bug a user would have hit: - The index line parsed the whole code-index metadata — every chunk's text — on every prompt composition, several times a turn. `CodeIndexSDK.is_indexed` answers the same question with two `exists()` calls. - A background index that died left the prompt saying "building now" for the rest of the session, so the model kept waiting for something that would never arrive. The trigger is now a four-state machine and inspects the tool's JSON, which is how `index_codebase` reports a refusal rather than raising. - The map told the model `run_shell_command` accepts `uv`, `npm` and `python`. It accepts none of them. Installed-and-allowlisted, installed-but-refused, and not-installed are now three separate lines — and the whole section is omitted for an agent that has no shell tool. - In dev mode the daemon launches the agent sidecar from the GAIA checkout, so the working directory resolved to GAIA's own source and it would have background-indexed itself. `is_agent_own_source` rejects that; an explicit `GAIA_PROJECT_ROOT` is exempt. - Listing `ProjectMapMixin` after the base agent silently disabled the index trigger while the prompt still rendered. `__init_subclass__` now raises. Also: the root resolves once per session so the map and the code index cannot describe two different trees, and the day-0 memory fact that duplicated `git`/`node`/`docker` (and carried an `except Exception: pass`) is gone — `DEV_TOOL_PROBES` feeds the map, which is all #3379 asked for. (cherry picked from commit 76d7849)
Skill audit
✅ All audited skills cleared the tier they claim. Per-finding detail is withheld here on purpose. Read it in the Security > Code scanning tab, or download the |
Request changes
The agent now opens a task knowing the repo's shape, entry points and installed commands instead of learning them one failed tool call at a time. The design is tight — a stated 600-token ceiling that's actually enforced and tested, a testable "is this a repository" predicate, and a class-definition guard that turns the one silent way to mis-wire it into a loud TypeError. Two narrow bugs stand between it and merge.
The map silently disappears for anyone who installs GAIA into their own project's virtualenv. The check that stops the agent from mapping its own source tree asks "is this directory an ancestor of the gaia package?" — and when GAIA is pip-installed into a .venv at the root of a user's project, that project is an ancestor. The user gets no map and no auto-indexing, and the log line tells them their repo is GAIA's source. This hits the embedded-SDK path the flagship guide documents. Narrow the check to a source checkout.
A project root that goes away mid-session takes the whole turn with it. The root is resolved once when the agent is built, then walked at the start of every task with no error handling. Delete, rename or unmount that directory and every subsequent query dies on a raw filesystem error. The prompt-rendering path already treats a failing map as droppable-with-a-warning; the task-start path should do the same. Orientation is a convenience — it should never be able to fail a query.
An eval comparison is still owed. This adds a system-prompt fragment and touches prompt assembly, which is exactly the class of change the repo requires gaia eval agent for. The PR notes the eval lane is red on an unrelated known issue with a fix in flight — so this is a "run it once that lands", not a reason to redesign anything.
Real-world evidence
An evidence bundle ran and is unusually honest about what it does and doesn't cover. Two surfaces were genuinely exercised on the CI runner:
$ printf 'y\n' | gaia memory bootstrap --system [system:software] Installed applications: Chrome, Firefox, git, Node.js, Docker, npm ✅ Stored 12 system context item(s). exit=0 $ curl -s -X POST -w "\nHTTP %{http_code}\n" http://127.0.0.1:4200/api/memory/refresh-system-context {"stored":12,"skipped":false} HTTP 200
That covers the refactor that routes the old per-tool shutil.which loop through the new shared probe — same 12 facts, same de-duplicated app labels, through both the real CLI and the real HTTP route. The flagship sidecar also still boots and answers its handshake from this ref (/health and /version both 200 under gaia daemon start-agent gaia --mode dev).
The feature itself was not exercised. The bundle says so plainly: gaia_agent.agent is imported lazily, so the new base order, the root resolution and the index rewiring are first reached on a query that needs a model. The rendered ==== PROJECT MAP ==== block, the not built → building → built status flip, and an Agent UI screenshot are all marked pending strix-halo lane — faiss is absent and there's no inference on the runner. That's the expected CI-lane deferral, not a gap, and the 64 unit tests cover the budget, the predicate, cache invalidation and the once-per-session trigger.
So: the evidence supports the refactor, and it neither supports nor contradicts the two bugs above — both were found by reading, and neither is something this lane could have caught. My verdict rests on static review for the map itself.
🔍 Technical details
Issues
🟡 is_agent_own_source false-positives on any project with GAIA installed in a local venv (src/gaia/agents/base/project_map.py:804)
package = Path(gaia.__file__).resolve().parent; for a wheel installed at <proj>/.venv/lib/python3.x/site-packages/gaia, package.parents contains <proj>, so is_agent_own_source("<proj>") is True. resolve_project_root then returns None immediately (project_map.py:852-859) — no map, no auto-index — and logs "...is GAIA's own source tree", which is not what happened. docs/guides/gaia.mdx:129 documents constructing GaiaAgent directly from a user's project, which is exactly this layout.
The docstring already states the real condition — "the daemon launches the agent sidecar with its working directory set to the GAIA checkout in dev mode". Only a source/editable checkout qualifies; an editable install still points gaia.__file__ at <repo>/src/gaia, so the dev-mode case keeps working:
import gaia
package = Path(gaia.__file__).resolve().parent
# Only a source/editable checkout is GAIA's "own" tree. A wheel installed
# into a venv under the user's project would otherwise make that project
# look like GAIA and silently cost it its map.
if any(p.name in ("site-packages", "dist-packages") for p in package.parents):
return False
path = Path(root).resolve()
return path == package or path in package.parents
🟡 An unreadable project root fails the turn, not just the map (src/gaia/agents/base/project_map.py:1100)
_on_task_start → materialize_project_map → build_project_map → _fingerprint, whose os.scandir(root) (project_map.py:650) and _collect's (project_map.py:712) are unguarded. The root is resolved once and cached for the session (_project_map_root_cache), so a root that is later deleted, renamed or unmounted raises FileNotFoundError/OSError out of _process_query_impl:4505 on every subsequent query — process_query wraps it in try/finally with no except.
Note the asymmetry: the rendering path is already safe, because _get_mixin_prompts:934 catches and logs a raising fragment. is_code_repository and _collect's subdir loop both guard OSError too — the two top-level scandir calls are the outliers. This is not a silent-fallback violation: the map is decorative and the failure is logged and named.
def _on_task_start(self, user_input: str) -> None:
"""Materialize the map and, if warranted, kick off ``index_codebase``."""
super()._on_task_start(user_input)
try:
pm = self.materialize_project_map()
except OSError as e:
# The root is resolved once per session and can vanish under us.
# Orientation is decorative — it must not take the turn with it.
logger.warning("[project-map] cannot read the project root: %s", e)
return
if pm is not None:
self._maybe_start_background_index(pm)
🟡 Eval comparison missing for a prompt-surface change
get_project_map_system_prompt is a new mixin prompt fragment and the change touches VOLATILE_PROMPT_FRAGMENTS / prompt-assembly order (src/gaia/agents/base/agent.py:615, 4505) — both are on CLAUDE.md's "REQUIRES an eval run before merge" list. The description acknowledges the lane is red on #3341 pending #3403. Re-run gaia eval agent against the committed baseline once #3403 lands and post the diff; no code change needed here.
Nits
_tool_names()returns the tool registry dict, not names (project_map.py:1056). Both call sites use it as a registry (.get("index_codebase"), membership on keys)._tool_registry()would stop the next reader from assuming a list of strings._MAX_ASCEND = 4reaches three ancestors, not four —[cwd, *list(cwd.parents)[:_MAX_ASCEND - 1]](project_map.py:847).docs/guides/project-map.mdxinherits the off-by-one ("up to four levels above it"). Either the constant or the prose should move.docs/guides/gaia.mdxis where a reader goes to configure the flagship (it documentsGaiaAgentConfigand theGAIA_*variables) but never mentions the map or links/guides/project-map, soproject_rootandauto_indexare discoverable only from the new page.
Strengths
ProjectMapMixin.__init_subclass__(project_map.py:1008) converts the one failure mode that would otherwise be invisible — wrong base order, prompt still renders, trigger silently never fires — into aTypeErrorat class definition, with the fix in the message. The AST test attests/unit/test_project_map.py:1825pins the wiring without importing through an editable install that may resolve elsewhere.CodeIndexSDK.is_indexed()(sdk.py:502) as a two-exists()presence check, with a docstring explaining whyget_status()is the wrong call here, is the right call for something rendered every turn — andtest_index_status_is_a_presence_check_not_a_metadata_parselocks it in.- Consolidating
shutil.whichintoprobe_binariesremoved threeexcept Exception: passblocks fromcollect_system_infowhile keeping the de-duplicated label behaviour, which the CLI evidence above confirms. test_render_stays_within_budget_on_a_pathological_repo(200 dirs ×ばつ 20 children) andtest_high_priority_sections_survive_truncationtest the budget as a guarantee rather than as a typical value.
The agent started every task blind to the project it was in. It did not know the directory shape, where the entry points were, or which programs were actually installed — so it guessed, and a meaningful share of its wasted steps were those guesses coming back as "no such file" or "command not found". It now gets a short, bounded orientation at the start of a task instead of discovering the same facts one failure at a time.
Closes #3379.
Test plan
python -m pytest tests/unit/test_project_map.py -q— 64 tests covering the budget, the repository predicate, caching and invalidationNotes for the reviewer
system_contextprobe rather than adding a second one. That probe already collected machine-level facts — OS, CPU, installed applications — but is opt-in, off by default, and lands in memory rather than the prompt. The genuinely new part is project shape and a per-task trigger.index_codebasea trigger. That tool has been composed on the flagship all along with nothing ever calling it automatically.A red
rag_quality + context_retention + tool_selectioncheck is expected on every PR right now — #3341, fixed by #3403, which has to merge before other branches see it.