feat(r5): runtime shape + causal-protocol behavioural contracts #7

Phase A plumbing for R5 runtime behavioural contracts. No call sites
wired yet — outbound and inbound paths still use run_pipeline. Adds:
- DialectCache: parses each install into a cbcl_core::dialect::Dialect
 and maintains an in-memory DialectRegistry alongside the textual
 cache. Exposes registry_snapshot() (clone) and with_registry(|r| ...)
 (borrow) for later PipelineContext construction.
- AgentEntry: per-handle Arc<Mutex<ThreadedMessageStore>> created on
 insert. AgentStore gains store_handle / append_message /
 lookup_message helpers. Per-handle, not global — each connection has
 its own causal world.
- CbclValidationError: new ShapeViolation / CausalViolation variants
 plus from_pipeline_validation factory. cbcl_validation_error_to_api
 maps them to 422 with codes "shape_violation" / "causal_violation".
 CLI exit-code dispatch adds both new codes to the CbclValidation arm
 (ExitCode 8).
- Tests: one in dialect_cache.rs verifying registry_snapshot contains
 the installed dialect; one in daemon.rs verifying append_message +
 lookup_message round-trip with dedup and unknown-handle paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Switch the outbound send path (reply/error/progress) from the lightweight
`run_pipeline` to `run_pipeline_full` with a per-agent runtime context.
Shape constraints (REQ-220, REQ-231 step 6b) and causal protocol
violations (REQ-200, REQ-231 step 6a) now surface as the dedicated
`shape_violation` / `causal_violation` API codes (HTTP 422, CLI exit 8)
instead of being lumped under `cbcl_malformed`. Successfully validated
messages are appended to the per-handle causal store before the WebSocket
write so subsequent replies in the same thread can resolve `:caused-by`.
- Moves the SPEC-009 dialect cache from a router-local variable onto the
 per-agent `AgentEntry` so the outbound HTTP handler (`local_api.rs`)
 and the router receive loop share the same cache instance. New
 `AgentStore::dialect_cache_handle` returns a cheap clone.
- Introduces `validate_for_send_with_context` which drives
 `run_pipeline_full` with the agent's registry snapshot and a mutable
 borrow of its `ThreadedMessageStore`. Falls back to `run_pipeline`
 when the outer `lang <name>` wrapper references a dialect the daemon
 has not been taught — fail-closed against an unknown dialect would
 reject otherwise-valid traffic on agents that advertised a name but
 haven't yet received the corresponding `(define ...)` push.
- Maps `PipelineResult::Pending` (default Reject policy: unknown causal
 predecessor) to `CausalViolation`; `Buffered` is treated as internal
 because hark never opts into Buffer policy. Existing `ParseError` and
 non-shape/non-causal `ValidationError` handling is preserved.
- Adds two focused unit tests in `cbcl_validation::tests` covering the
 new error codes and one fallback test confirming unknown `lang`
 dialects still pass.
Phase A error mappings (`shape_violation` / `causal_violation` →
HTTP 422 / CLI exit 8) and the `from_pipeline_validation` factory are
now exercised end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Phase C: gate the router receive loop's `process_inbound` on
`run_pipeline_full` for `InboundClass::Ordinary` frames. Pipeline runs
against the per-handle dialect cache's `DialectRegistry` snapshot and the
per-handle `ThreadedMessageStore` (locked across the synchronous call).
On `ValidationError::ShapeViolation` / `CausalViolation` or `Pending`,
emit a structured `tracing::warn` (`target = "hark::r5"`, performative,
thread, blame) and drop the frame before `enqueue_inbound`. Other
pipeline outcomes — parse errors, malformed messages, R1–R4 meta
envelope failures, eval errors like `UnknownDialect`, and `Buffered` —
fall through to the existing enqueue path; Phase C's contract is shape
+ causal enforcement only.
On `Success`, compute the canonical content hash of the innermost simple
message (`sha256:<hex>` of `canonical_encode(SExpr::from(simple))`) and
append `(hash, thread, message)` into the agent's causal store before
enqueueing, so subsequent inbound frames in the same thread can resolve
this message as a `:caused-by` predecessor.
Meta pushes (already R1–R5-checked at install) and meta-replies that
fell through the meta-reply correlator skip the new gate.
Adds one unit test that installs a `(shape greet (require :target
string))` dialect and feeds an inbound `(greet :name ... :thread ...)`
frame missing `:target`, asserting `process_inbound` returns
`Continue` without enqueueing (recv times out).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Adds six end-to-end tests under `tests/r5_runtime.rs` covering the full
daemon path for outbound (`local_api::send`) and inbound
(`router::process_inbound`) R5 enforcement:
- outbound shape-violation rejected with HTTP 422 shape_violation
- outbound causal-violation rejected with HTTP 422 causal_violation
- outbound happy path forwards to the router and appends to the store
- inbound shape-violation dropped (no enqueue)
- inbound causal-violation dropped (no enqueue)
- inbound happy path enqueues and appends to the store
The shared fixture installs a tiny R1-R5-clean `greet-d` dialect with a
`(then begin (any greet reply))` protocol and a `(shape greet (require
:target string))` rule. Tests drive a controllable mock router that
exposes an explicit `dispatch(frame)` so the dialect can be installed
into the per-handle cache before inbound frames arrive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The SPL plan that orchestrated the infra/outbound/inbound/tests/docs
phases of the R5 behavioural-contracts implementation. Kept for
historical traceability; it has been fully completed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Three follow-ups from the live cbcl-router smoke test:
1. main.rs now initialises a tracing_subscriber::fmt writer to stderr with
 an EnvFilter (default `info`, overridable via RUST_LOG). Without this
 the Phase C `tracing::warn!(target: "hark::r5", ...)` events on inbound
 R5 violation were silently dropped and operators had no visibility into
 the per-frame drop policy.
2. local_api.rs `meta_publish` now installs the published `(define ...)`
 form into the publishing handle's local dialect cache after the router
 acks. Previously a successful publish left the publisher's own R5
 constraints unenforced on subsequent outbound traffic — the agent had
 to round-trip through `hark dialect query <name>` to pick up its own
 newly-published dialect. A local install failure is non-fatal and
 surfaced via a warn-level event under target `hark::dialect_cache`.
3. specs/router-protocol.md documents (a) that publish now installs
 locally on router ack, and (b) the conjunction-over-all-installed-
 dialects scope: an unwrapped `(reply ...)` is still checked against any
 installed dialect whose protocol mentions `reply` (cbcl-rs REQ-231,
 complete monitoring).
Verified end-to-end against a local cbcl-router on :18080: publishing
`greet-d` with `(protocol (then begin (any greet reply)))` and immediately
attempting `(lang greet-d (reply @router "ok" :thread "t-1"))` (no
`:caused-by`) now correctly returns HTTP 422 `causal_violation` from the
same handle that published the dialect.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Verified the CLI spec against the actual binary and the runtime R5 paths.
Three drifts fixed in cli.md, one in daemon.md:
* `init` documented `--capability` and a JSON shape with `capabilities`,
 but SPEC-009 collapsed capability ≡ dialect and the CLI only accepts
 `--dialect`. Updated the example, JSON sample, options list, and the
 duplicate-rejection clause to match. Same fix in daemon.md's init
 example.
* `daemon status` documented printing "capabilities" alongside queue
 sizes; the actual status output prints advertised dialects. Updated
 the line.
* `recv` made no mention of inbound R5 drops. Added a paragraph
 documenting that shape/causal violations on router-dispatched frames
 are silently dropped before reaching the recv queue, never transition
 the handle to an error state, and surface only via `tracing` events
 under target `hark::r5`. Cross-references router-protocol.md for the
 policy details.
* `dialect publish` did not say it installs the published define into
 the publishing handle's local cache on router ack. Added a note about
 this and the non-fatal `hark::dialect_cache` warn behaviour, matching
 the fix in 7f34dbd.
Broader SPEC-009 drift (capability terminology in daemon.md, config.md,
local-api.md, router-protocol.md) is pre-existing and orthogonal to R5;
left for a separate spec-cleanup pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Records the final claim/complete state for the polish task from the SPL
coordination plan. Plan is fully retired; kept in the tree for historical
traceability of the work breakdown.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Two small alignments after commit 7f34dbd:
* The architecture blurb listed `dialect query` and `subscribe` as the
 paths that get a dialect into the per-handle registry. Publish now
 also installs locally on router ack, so it belongs in that list — and
 agents that publish a dialect should be subject to their own
 constraints immediately.
* The `dialect publish` per-command section now documents the local
 install side effect and the non-fatal `hark::dialect_cache` warn
 behaviour on install failure after a successful router ack.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Closes the init/advertise gap surfaced by the live-router smoke test:
previously, `hark init --dialect greet-d` would advertise greet-d to the
router but leave the per-handle DialectCache empty, so the R5 pipeline
fell back to the lightweight R1-R4 path until the agent separately
called `dialect publish` or `dialect query`. A new agent could
advertise a capability the daemon couldn't actually enforce.
After the router hello, the create-agent handler now issues a
best-effort `(meta (query (speak? <name>)))` per advertised dialect.
The existing receive-loop teach-back path installs each result into
the per-handle cache, so R5 fires from the very first message on hit.
Behaviour matrix on miss / timeout / transport error (per dialect):
* `PushInstalled`: info-log at `hark::auto_install`; cache populated.
* `Reply` (router does not know the dialect — publisher-first flow):
 info-log; init proceeds; fallback applies until publish or push.
* `PushInstallFailed`: warn-log; init proceeds.
* `Err(_)` from `send_meta_and_await` (timeout, busy, etc.):
 warn-log; init proceeds.
Soft-fail is deliberate: a publisher who advertises a dialect they are
about to author must be able to init even when the router has never
heard of it. Per-dialect timeout is 1.5s (vs the 10s used elsewhere)
so misses on slow / non-responsive routers don't make init feel laggy.
Knobs:
* `[agent].auto_install_advertised` config flag (default `true`).
* `CBCL_AGENT_AUTO_INSTALL_ADVERTISED=false` env override.
* `auto_install_advertised: false` per-request override on the
 local-API create-agent JSON body. Wins over the daemon default.
Tests:
* `tests/support/mod.rs` (process-spawned daemons) sets the env var
 to false so mock routers that don't speak meta queries don't stall
 init.
* `tests/router_integration.rs` and `tests/r5_runtime.rs` (in-process
 daemons with mock routers) pass the per-request override on each
 agent creation.
* All 142 tests still pass. Verified end-to-end against a live
 cbcl-router: pre-publish greet-d on one handle, close, then init a
 fresh handle with `--dialect greet-d` (no publish/query), and the
 very next outbound reply without `:caused-by` correctly returns 422
 causal_violation — the new handle picked up greet-d's protocol via
 the auto-install handshake.
Docs: router-protocol.md, cli.md, README.md updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

P2 — Refresh registry entries on dialect updates (dialect_cache.rs):
Before this change `latest_by_name` was updated on a same-name re-install
(e.g. publishing v2 of greet-d after v1) but the DialectRegistry kept
holding v1's parsed Dialect, so registry_snapshot() returned the stale
shape/protocol rules. The R5 pipeline then enforced v1 even though the
textual cache resolved to v2's digest.
Fix: drop the live `registry` field from Inner and rebuild the
registry lazily from `latest_by_name → parsed_by_digest` on demand,
memoising the result in `cached_registry`. Any mutation
(`install_unchecked_with_dialect`) clears the memo so the next
snapshot picks up the new parsed Dialect. Install order is preserved
via a new `install_order: Vec<String>` so child dialects that
`(extend ...)` a custom parent get installed after their parent on
rebuild.
Regression test `same_name_update_refreshes_registry_snapshot`
installs a v1 with no protocol, then a v2 with one, and asserts the
second snapshot's Dialect.causal_protocol is Some(_).
P2 — Serialize store appends with outbound writes (local_api.rs send):
Before this change the send handler took the store lock to validate,
released it to append, released it again before calling
`send_outbound`. With concurrent /send requests on the same handle,
sender A could pass validation + append its hash, and sender B could
then pass validation against that hash and reach send_outbound first
— the writer task pops the mpsc in FIFO order, so B's frame would
leave the WebSocket before its predecessor A.
Fix: hold the per-handle store guard across the entire
(validate, append, enqueue) sequence. The mpsc writer preserves
enqueue order, so the wire order now strictly follows the store-append
order. Lock ordering: store guard taken first, agents lock acquired
internally by `send_outbound` — no other path holds those in reverse.
P3 — Return R5 blame fields in error responses (local_api.rs):
Spec advertised `performative` and `thread` alongside `code`/`message`
in the 422 body for shape_violation and causal_violation, but the
mapping arms discarded those fields from CbclValidationError and
ErrorBody only serialised `code/message/hint`. Clients never saw the
blame context.
Fix: add optional `performative` and `thread` fields to `ErrorBody`
(skip_serializing_if = Option::is_none so non-R5 codes stay
wire-compatible), and `ApiError::with_blame` to attach them; the
ShapeViolation / CausalViolation arms now plumb the fields through.
Regression test
`outbound_r5_error_body_includes_performative_and_thread_blame`
posts a shape-violating greet and asserts both fields are present.
Verified end-to-end against a live cbcl-router:
 curl ... /v1/agents/$HANDLE/send -d '{"kind":"reply", "message":
 "(lang greet-d (reply @router \"ok\" :thread \"t-blame-live\"))"}'
 → 422 {"error": {"code": "causal_violation",
 "message": "causal violation: missing :caused-by",
 "performative": "reply",
 "thread": "t-blame-live"}}
Tests: 144 pass (was 142; +1 same-name update, +1 blame-fields).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

P1 from PR #7 review: the previous P2-2 fix held the per-handle causal
store mutex across `send_outbound`'s writer-ack await. The router
receive loop's `tokio::select!` runs outbound writes AND inbound
`verify_inbound_against_registry` in the same task; the inbound branch
takes the same store mutex. When `tokio::select!` picked the inbound
branch while /send was awaiting its ack:
 * inbound was blocked on the store mutex held by /send,
 * the receive-loop task was therefore parked,
 * the outbound branch (the only path that delivers the ack) never
 ran,
 * /send waited forever for `result_rx`.
Both directions wedged on the handle.
Fix: introduce a per-handle `send_sequencer: Arc<Mutex<()>>` on
`AgentEntry`, distinct from the store mutex and **never taken by the
receive loop**. The /send handler now:
 1. Acquires the sequencer (serialises concurrent /send on this
 handle).
 2. Acquires the store mutex.
 3. Validates, appends, drops the store mutex.
 4. Calls `send_outbound` (enqueue + await ack) with the store
 mutex released. The receive loop's inbound branch can now run
 `verify_inbound_against_registry` freely while we wait for the
 ack.
 5. Drops the sequencer.
This preserves the property the P2-2 fix was after — concurrent
senders agree on a wire order that matches the store-append order,
because the (validate, append, enqueue) sequence remains serialised
on the sequencer — while removing the cross-arm deadlock.
Lock ordering invariants:
 * Only the /send handler takes `send_sequencer`. Neither the
 receive loop nor any other path does. No deadlock from
 sequencer-vs-anything.
 * /send takes `send_sequencer` then `store` (briefly). The receive
 loop takes `store` only. No reversed ordering.
 * `send_outbound` (enqueue + ack-await) runs with no per-handle
 locks held by the caller.
Regression test `concurrent_outbound_and_inbound_do_not_deadlock`
runs on a multi-threaded tokio runtime, strictly interleaves
/send + router.dispatch + recv across BURST=8 iterations, and
asserts the whole sequence completes within 5s. Verified by
temporarily reverting the fix: the test reliably wedges and fails
with `Elapsed(())` after the inner timeout. Verified again with the
fix in place: passes in ~20ms.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>