×ばつ20 directory tree. "Code repository" predicate: is_code_repository(path) — a directory from VCS_DIRS (.git, .hg, .svn) or a file from PROJECT_MANIFESTS (14 entries) at the root. Non-recursive, so a home directory full of repositories is not itself one. Every manifest in the list is parametrized in the tests. Platform quirks: exactly three, the fields of PlatformQuirks — path separator, path quoting for spaces, shell dialect. A test asserts the dataclass has those three fields and no others, so the list stays closed. The binary probe was extended, not duplicated. probe_binaries() is now the single PATH probe in the codebase; collect_system_info and the map both go through it. The extension over the seven commands it already detected (git, code, cursor, node, docker, brew, npm — chosen because they double as desktop-app markers) is DEV_TOOL_PROBES: 29 build, package, runtime and VCS binaries. The map also crosses the result with run_shell_command's own allowlist, so it can say up front which commands the shell tool will refuse instead of letting the agent discover that one refusal at a time. Two behaviours worth a reviewer's attention Auto-indexing costs something on first contact. When the root is a repository with no index, the map starts index_codebase in a background thread. On a large repo that is minutes of local embedding, and the embedder can evict the resident chat model — the same trade the RAG warm-up already makes, so the cost is first-turn latency, not a wrong answer. GAIA_PROJECT_MAP_AUTO_INDEX=0 turns it off. It fires at most once per session, and a failure is reported in the prompt as build FAILED rather than leaving the model waiting on something that will never arrive. Root resolution refuses to point at GAIA itself. In dev mode the daemon launches the agent sidecar with its working directory set to the GAIA checkout, so a naive cwd rule would have had the flagship map — and background-index — its own source tree. is_agent_own_source rejects that. An explicit GAIA_PROJECT_ROOT is exempt: pointing GAIA at GAIA is legitimate when you mean it. Eval not run: this changes the system prompt, which CLAUDE.md flags as eval-affecting. #3341 tracks gaia eval agent failing repo-wide because the API account behind ANTHROPIC_API_KEY is out of credit. Test plan python -m pytest tests/unit/test_project_map.py -q — 64 tests: the predicate against all 14 manifests, the budget on a ×ばつ20-directory repo, cache invalidation on directory/manifest/PATH change, the four index-trigger states, and the MRO guard. python -m pytest tests/unit/ -q — no new failures against the pre-change baseline (616 failed / 9,621 passed before, 614 / 9,679 after; the deltas are the new tests). The email- and hub-agent modules fail identically before and after for an unrelated editable-install reason. python util/lint.py --all — black, isort, pylint, flake8, bandit and the agent-convention checks pass. Print the real map for this repo and confirm it is under budget and factually correct: GAIA_PROJECT_ROOT=$(pwd) python -c "from gaia.agents.base.project_map import *; from gaia.agents.base.turn_metrics import count_tokens; t=render_project_map(build_project_map(resolve_project_root())); print(t); print(count_tokens(t))" Expect ~470 tokens, and every name on the run_shell_command accepts: line present in ALLOWED_COMMANDS. From a directory that is neither a VCS checkout nor holds a manifest, confirm resolve_project_root() returns None and the prompt fragment is empty.">
Skip to content

Navigation Menu

Sign in
Sign up

feat(agent): materialize a project map at task start #3404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kovtcharov-amd wants to merge 2 commits into main
base: main
Choose a base branch
Loading
from claudia/task-9659f330
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/docs.json
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,8 @@
{
"group": "Developer Tools",
"pages": [
"guides/code-index"
"guides/code-index",
"guides/project-map"
]
},
{
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/code-index.mdx
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,12 @@ The [`coding` skill](https://github.com/amd/gaia/blob/main/hub/skills/coding/SKI
| `get_index_status` | Report current index state |
| `clear_code_index` | Remove the cached index |

You usually will not have to wait for that first `index_codebase` call. The
[project map](/guides/project-map) starts one in the background at task start
whenever the agent is in a repository that has no index, so by the time a
semantic search matters the index is often already there. Set
`GAIA_PROJECT_MAP_AUTO_INDEX=0` to keep indexing entirely manual.

### Example interaction

```
Expand Down
153 changes: 153 additions & 0 deletions docs/guides/project-map.mdx
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
---
title: "Project Map"
description: "The orientation pass the agent runs at task start, so it stops guessing paths and commands."
icon: "map"
---

<Info>
**Source Code:** [`src/gaia/agents/base/project_map.py`](https://github.com/amd/gaia/blob/main/src/gaia/agents/base/project_map.py)
</Info>

Without a project map the agent opens every task blind. It guesses directory
names, guesses which programs are installed, and guesses which shell it is
talking to — and every wrong guess costs a full round trip to learn something
one orientation pass establishes once.

The [GAIA agent](/guides/gaia) now runs that pass at the start of every task and
puts the result in its system prompt.

## What it contains

```text
==== PROJECT MAP ====
Root: C:\Users\me\Work\gaia
Code repository: yes (pyproject.toml, setup.py, package.json)

Platform (these three change the commands you write):
- Path separator: \
- Paths with spaces: wrap in double quotes: "C:\Program Files\app"
- Shell dialect: cmd.exe

Directories:
- docs/ (assets, deployment, guides, integrations)
- hub/ (agents, components, skills)
- src/ (gaia, vscode)
- tests/ (electron, fixtures, integration, mcp, unit)

Entry points: Makefile, src/cli.py, npm run build

NOT installed, do not invoke: cargo, gradle, java, mvn, rustc
run_shell_command accepts: cat, date, diff, find, git, grep, head, ls, ...
Installed but run_shell_command refuses them — use a tool, not the shell:
cmake, curl, docker, go, node, npm, pip, python, uv

Code index: not built — call index_codebase to enable semantic code search
```

| Section | Where it comes from |
|---------|---------------------|
| Root, repository status | The `is_code_repository` predicate below |
| Platform | `detect_platform_quirks()` — exactly three fields, see [Platform quirks](#platform-quirks) |
| Directories | Two levels deep, build output and vendored dependencies skipped |
| Entry points | A closed list of well-known files, plus `npm run` targets from `package.json` |
| Commands | `probe_binaries()` — the same PATH probe that backs [day-0 memory](/guides/memory), crossed with the shell tool's own allowlist |
| Code index | [`CodeIndexSDK.is_indexed()`](/guides/code-index) — a presence check, so it is cheap enough to run on every render |

## The token budget

**600 tokens.** That is 1.8% of the NPU profile's 32,768-token window
(`NPU_CTX_SIZE`) and 0.9% of the GPU profile's 65,536. It is sized against the
smaller window on purpose — a budget that only holds on 64K is not a budget.

Measured with GAIA's shared estimator (`count_tokens` — cl100k, or a character
ratio when tiktoken is absent), so it bounds the map to within that estimator's
error of the model's own count rather than to the exact token.

`render_project_map()` enforces it on every render. Sections are emitted in
priority order and the directory listing carries its own sub-cap, so on a
2,000-directory monorepo the platform quirks and the command list still survive
while the tree is what gets trimmed.

## Which directories count as a code repository

The predicate is testable, not a judgement call. **`is_code_repository(path)` is
true when a version-control directory or a recognised manifest sits at the
root** — non-recursive, so a home directory full of repositories is not itself
one.

<AccordionGroup>
<Accordion title="Version-control directories">
`.git`, `.hg`, `.svn`
</Accordion>
<Accordion title="Recognised manifests">
`pyproject.toml`, `setup.py`, `setup.cfg`, `requirements.txt`,
`package.json`, `Cargo.toml`, `go.mod`, `pom.xml`, `build.gradle`,
`build.gradle.kts`, `CMakeLists.txt`, `Makefile`, `Gemfile`,
`composer.json`
</Accordion>
</AccordionGroup>

## Platform quirks

Three, and only three — the differences that change the *text* of a command the
model emits. A contributor can check the list off:

1. **Path separator** — `\` on Windows, `/` elsewhere.
2. **Path quoting for spaces** — double quotes on Windows, single quotes on POSIX.
3. **Shell dialect** — `cmd.exe` or PowerShell on Windows; `$SHELL`'s basename otherwise.

## Automatic code indexing

When the root is a code repository and no [code index](/guides/code-index)
exists, the map calls `index_codebase` in a background thread and says so in the
prompt, so the model knows to grep until it lands. It fires at most once per
session.

Turn it off on a monorepo where a full embedding pass is not worth it:

```bash
export GAIA_PROJECT_MAP_AUTO_INDEX=0
```

## Choosing the root

Resolution order:

1. `project_root` on `GaiaAgentConfig`
2. the `GAIA_PROJECT_ROOT` environment variable
3. the working directory, or the nearest repository up to four levels above it

A working directory that resolves to **GAIA's own source tree** is rejected — in
dev mode the agent sidecar is launched from the GAIA checkout, and mapping (and
auto-indexing) its own source is never what the user asked for. An explicitly
configured root is exempt: pointing GAIA at GAIA is fine when you mean it.

If none of those is a repository the agent gets **no map at all**. That is the
right answer, not a degraded one — an agent answering questions from a home
directory is not in a project, and a map of `~` would spend tokens describing
nothing.

```bash
export GAIA_PROJECT_ROOT=/home/me/Work/my-service
```

A `GAIA_PROJECT_ROOT` that does not exist raises at startup rather than being
quietly ignored.

## Caching

The map is built once per root and reused. It is rebuilt when a fingerprint over
four things changes: the top-level directory listing, every manifest's size and
mtime, the VCS head, and `PATH`. Between changes, repeated queries reuse one
walk.

## Related

<CardGroup cols={2}>
<Card title="Code Index" icon="magnifying-glass-code" href="/guides/code-index">
The semantic index the map triggers.
</Card>
<Card title="Memory" icon="brain" href="/guides/memory">
Day-0 system facts, from the same binary probe.
</Card>
</CardGroup>
8 changes: 8 additions & 0 deletions hub/agents/gaia/npm/CHANGELOG.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@ the terminal UI meant building it from source.

### Added

- **A project map at task start.** In a code repository the agent now opens
every task knowing the directory shape, the likely entry points, which
commands are installed, and the three platform differences that change
command syntax — instead of discovering each one through a failed tool call.
Capped at 600 prompt tokens. If the repository has no code index the map
starts one in the background; `GAIA_PROJECT_MAP_AUTO_INDEX=0` turns that off,
and `GAIA_PROJECT_ROOT` picks the project when the working directory is not
it. See SKILL §11.
- **`503` from `/query` at session capacity.** When every retained session
slot is busy and none is idle enough to evict, starting a new session
returns `503` with the reason in `detail` — retryable, distinct from a
Expand Down
29 changes: 27 additions & 2 deletions hub/agents/gaia/npm/SKILL.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,32 @@ packaged sidecar (its CLI accepts only `--host` and `--port`), and an undeclared
name raises naming the valid sets rather than falling back to a default. Beyond
`gaia-voice`, do not design around a skill being on by default.

## 11. Ports
## 11. The project map — two things it costs you

When the agent's working directory is a code repository, every task starts with
a **project map** in the system prompt: the root, the directory shape, the
likely entry points, which commands are installed, and the three platform
differences that change command syntax. It exists so the agent stops burning
round trips on "no such file" and "command not found".

Two consequences an integrator needs to plan for:

- **Up to 600 prompt tokens, every turn.** That is the enforced ceiling
(1.8% of the NPU profile's 32K window), not a typical value — budget it
alongside `gaia-voice`'s 676.
- **A background embedding pass on first contact with a new repository.** If
the repo has no [code index](https://amd-gaia.ai/docs/guides/code-index), the
map starts one in a background thread so semantic search is ready when it is
needed. On a large monorepo that is minutes of local embedding.
`GAIA_PROJECT_MAP_AUTO_INDEX=0` turns it off.

The sidecar's CLI accepts only `--host` and `--port`, so pointing the map at a
specific project means `GAIA_PROJECT_ROOT=/path/to/repo` in its environment, or
`GaiaAgentConfig(project_root=...)` when embedding. A directory that is neither
a VCS checkout nor holds a recognised manifest gets **no map** — that is the
designed answer, not a failure.

## 12. Ports

| Service | Port |
|---|---|
Expand All @@ -425,7 +450,7 @@ Port **4001 is reserved repo-wide**: `spawnSidecar` throws a `RangeError` and
speaks for the user's documents and memory and has no business on a LAN
interface.

## 12. Running in a server or long-lived app
## 13. Running in a server or long-lived app

- **`fetchAll` / `fetchBinary` are a build step**, not per request — network plus
a full SHA-256 hash of a large artifact. Run once at install time.
Expand Down
9 changes: 9 additions & 0 deletions hub/agents/gaia/npm/SPEC.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,15 @@ collapses the rest to a one-line menu entry (re-activated by calling
disables it for the session and every body renders. Omitting it is a valid, explicit one-shot: nothing
persists past that single turn, and the agent is not told otherwise.

A retained session also carries a **project map** — up to 600 prompt tokens of
directory shape, entry points, installed commands and platform quirks, present
whenever the agent's working directory resolves to a code repository (a VCS
directory or a recognised manifest at its root). `GAIA_PROJECT_ROOT=<path>`
picks the project when the working directory is not it. If that repository has
no code index, the first turn starts one in a background thread;
`GAIA_PROJECT_MAP_AUTO_INDEX=0` disables that. Neither affects the wire
contract — they change what the agent knows and what the first turn costs.

A second `/query` for a `session_id` that already has a turn in flight gets
`409 Conflict` — cancel the running turn or wait for it, then retry. A
`/query` that needs a **new** session while every retained slot is busy and
Expand Down
36 changes: 27 additions & 9 deletions hub/agents/gaia/python/gaia_agent/agent.py
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@

from gaia_agent_chat.agent import ChatAgent, ChatAgentConfig

from gaia.agents.base.project_map import ProjectMapMixin
from gaia.agents.base.skill_discovery import (
DISCOVERY_THRESHOLD_ENV,
SkillDiscovery,
Expand Down Expand Up @@ -204,11 +205,24 @@ class GaiaAgentConfig(ChatAgentConfig):
default_factory=lambda: [str(Path.home())]
)

# The project this task is about (#3379). ``None`` resolves through
# ``GAIA_PROJECT_ROOT``, then the working directory or the nearest
# repository above it — and stays ``None`` when neither is a repository,
# which is the common case for a sidecar launched from its package dir.
project_root: Optional[str] = None

# Base agent first, tool mixins after — the repo's MRO convention for every
# hub agent. Neither mixin overrides anything today; this order keeps a future
# mixin method from silently winning over ChatAgent's.
class GaiaAgent(ChatAgent, SkillLibraryToolsMixin, CodeIndexToolsMixin):
# Build the semantic code index at task start when the project is a
# repository and has none. Off-switch: ``GAIA_PROJECT_MAP_AUTO_INDEX=0``,
# for a monorepo where a full embed pass is not worth it.
auto_index: bool = True


# ``ProjectMapMixin`` is the one exception to "base agent first": it overrides
# ``_on_task_start`` and calls ``super()``, and ``Agent``'s no-op default sits
# ahead of every trailing mixin in the MRO — listed after ChatAgent it would
# never run. Tool mixins keep their usual place at the back, where neither
# overrides anything and a future method cannot silently win over ChatAgent's.
class GaiaAgent(ProjectMapMixin, ChatAgent, SkillLibraryToolsMixin, CodeIndexToolsMixin):
"""The flagship GAIA agent — conversation, documents, data, web, and skills."""

SKILL_DIRS: ClassVar[List[str]] = _bundled_skill_roots()
Expand Down Expand Up @@ -264,12 +278,16 @@ def _register_tools(self) -> None:
self.skill_loader = self._maybe_build_skill_loader()
self._skill_discovery = self._maybe_build_skill_discovery()
self.register_skill_library_tools()
# Same scope as allowed_paths, for the same reason that field rejects
# cwd: the daemon launches this sidecar with cwd = the package
# directory, so cwd would sandbox code search to the agent's own
# source tree — and index it by default.
# The project map's root when there is one, so "is the index built?"
# and "index it" both mean the repository the task is about. Falling
# back to allowed_paths for the same reason that field rejects cwd: the
# daemon launches this sidecar with cwd = the package directory, so cwd
# would sandbox code search to the agent's own source tree.
allowed = getattr(self.config, "allowed_paths", None) or [str(Path.home())]
self._init_code_index_state(repo_path=allowed[0])
# Through the mixin, so both read the one cached resolution and can
# never end up describing two different trees.
index_root = self._project_map_root() or allowed[0]
self._init_code_index_state(repo_path=index_root)
self.register_code_index_tools()
super()._register_tools()

Expand Down
18 changes: 18 additions & 0 deletions src/gaia/agents/base/agent.py
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -612,6 +612,9 @@ class Agent(abc.ABC):
"get_memory_system_prompt", # changes on any remember()/forget()
"get_skills_system_prompt", # per-turn body selection (#2848)
"get_recalled_skills_system_prompt", # per-turn procedural recall
# Mostly static, but the index line flips as a background index
# lands and the shape line changes if the project does (#3379).
"get_project_map_system_prompt",
}
)

Expand Down Expand Up @@ -1248,6 +1251,17 @@ def _select_tools_for_turn( # pylint: disable=unused-argument
"""
return None

def _on_task_start( # pylint: disable=unused-argument
self, user_input: str
) -> None:
"""Hook run at the top of every turn, before the prompt is composed.

Default: no-op. Mixins that orient the agent in its environment — the
project map (#3379) is the first — override this and must call
``super()._on_task_start(user_input)``. Anything expensive belongs
behind a once-per-session guard inside the override, not here.
"""

def _on_tool_invoked(self, tool_name: str) -> None:
"""Hook called when a tool is about to execute (after registry lookup).

Expand Down Expand Up @@ -4486,6 +4500,10 @@ def _process_query_impl(
self._current_query = user_input
self._single_tool_done = False

# Orientation. Runs before the prompt is composed so anything it
# establishes is in the prompt on the turn that established it.
self._on_task_start(user_input)

# Proactive skill discovery: a skill the user never named can become
# loaded here, registering its tools — so it must run BEFORE the tool
# filter, or those tools are invisible on the very turn that loaded the
Expand Down
Loading
Loading

AltStyle によって変換されたページ (->オリジナル) /