title: Automated Observability category: concepts tags: [harness, observability, metrics, layer-5, quality] status: developing created: 2026年04月28日 updated: 2026年04月28日 sources:

• "harness-implementation-plan" related:
• "agentic-harness"
• "adversarial-verification"
• "persistent-memory" layer: "Layer_5" summary: Layer 5 enforces that every subtask output includes an ObservabilitySpec — metrics, health checks, and alert conditions. If require_metrics is enabled (default), the subtask is not complete until instrumentation code exists in the change. provenance: extracted: 0.85 inferred: 0.15 ambiguous: 0.0

Netflix's chaos engineering and Google's SRE principles: build operability in from the start, not as an afterthought. Observability is part of the definition of done.

subtask_verified (critics passed)
 ↓
AI call: extract metrics, health checks, alerts from code change + task spec
 ↓
Produce ObservabilitySpec
 ↓
If require_metrics=true (default):
 → Scan code changes for instrumentation matching MetricDefinitions
 → If missing: generate scaffolding, block until wired
 ↓
Store ObservabilitySpec with SubtaskResult
 ↓
subtask_observable → mark task as done

• metrics — each with name, type (counter/gauge/histogram), description, unit, instrumentation_code
• health_checks — each with name, method, expected_result
• alert_conditions — each with metric_name, threshold, action

Every metric must answer "is this working correctly?" — no vanity metrics (like request_count without context).

The system scans code changes for:

• metric.name as string literal
• metric.type usage patterns (counter/gauge/histogram)
• instrumentation_code pattern matching

If require_metrics=true: missing instrumentation blocks completion. If require_metrics=false: only a warning is logged.

Note: This is heuristic — it catches obvious omissions, not compile-time guarantees. require_metrics defaults to true because observability is definition-of-done. Setting it to false is a deliberate concession, not an opt-out of the layer.

{
 "observability": {
 "require_metrics": true
 }
}

• lib/harness-observability.ts — ObservabilityEnforcer class
• extensions/harness-observability.ts — Extension with enforcement hooks