observability: no tenant on a trace event, so per-tenant cost cannot be attributed #8

Summary

TraceEvent in grapharc/observe/trace.py has no tenant field. Because grapharc.observe derives every number from that one file and nothing else, per-tenant cost attribution cannot be offered at all — it is not approximated, it simply does not exist. Meanwhile grapharc/policy/engine.py is already tenant-aware: permission_policy(tenant=...), edge_policy(tenant=...), record_spend(..., tenant=...), committed_usd(tenant=...) and a DEFAULT_TENANT.

So the policy layer can scope a rule to a tenant and refuse spend on its behalf, and the audit trail cannot say which tenant a run belonged to.

Why this matters

"Who spent this?" is the first question anyone asks of a multi-tenant deployment, and the second is "prove it." Cost attribution you can defend per tenant is what lets graph engineering carry a real bill rather than an estimate, and the deliberate design here — one JSONL file as the only writer, every reader derived from it — means the field has to be added at the source or not at all.

Where in the code

• grapharc/observe/trace.py — TraceEvent, and the record / event methods
• grapharc/observe/cost.py — attribute, attribute_thread, by_node, tokens_by_model
• grapharc/observe/metrics.py, replay.py, otel.py — the other readers of the same file
• grapharc/policy/engine.py — DEFAULT_TENANT, record_spend, committed_usd, the existing tenant model to stay consistent with
• grapharc/runtime/graph.py — RunContext, the natural carrier for a tenant through a run

Confirm it:

grep -n "tenant" grapharc/observe/trace.py # no hits
grep -n "tenant" grapharc/policy/engine.py | head

What to change

The mechanical part is small; the compatibility and semantics are not.

1. Backward compatibility is a hard constraint. TraceEvent already documents that record excludes None, so a trace from a producer that does not report a field is byte-identical to one written before that field existed, and old traces still parse. A tenant field must preserve that exactly — old files must keep loading, and a run with no tenant must not start writing a new key.
2. Where does the tenant come from? RunContext is the likely carrier, threaded from invoke(..., tenant=...), a config value, or the policy engine's tenant. Pick one and justify it; a tenant that can be set per-node rather than per-run would be a much bigger claim.
3. Sub-runs. The governed loop runs each round as a sub-run with its own meter and thread id. A tenant must propagate there or per-tenant totals will silently miss planner spend.
4. Reader API. Decide the surface: attribute_tenant(...), or a tenant= filter on the existing functions. Keep it consistent with attribute_thread's shape.
5. Recorded vs estimated must stay separated. recorded_cost_usd is never a guess today. Per-tenant reporting must not blur a provider-reported figure with a RateCard estimate.
6. The CLI. grapharc metrics / a cost command should be able to scope by tenant, including in --json.
7. Update the README bullet that currently says per-tenant attribution is not offered.

How to verify

uv run pytest tests/test_replay.py -q
uv run pytest -q
uv run ruff check .

Required tests: an old trace file with no tenant key still parses and reports; two tenants in one file are attributed separately and sum to the run total; planner sub-run spend lands under the right tenant.

Acceptance criteria

• Per-tenant token and cost attribution is derivable from the trace alone
• Traces written before this change still parse, and an untenanted run writes no new key
• Sub-run (planner round) spend is attributed correctly
• Recorded and estimated figures remain strictly separated
• The reader API is consistent with attribute_thread
• README's "no tenant on a trace event" bullet is updated
• uv run pytest green, uv run ruff check . clean

Skill level — experience required

The edit to TraceEvent is three lines; getting it right is not. This touches the single source of truth that five other modules derive from, and the format has an explicit backward-compatibility contract you must not break. You need to understand how replay reconstructs a run, how the governed loop's sub-runs are metered, and why recorded and estimated costs are kept apart. Please propose the tenant-propagation design in a comment before implementing.