Most "microservice projects" prove that an app works. TraceForge proves that a system can be deployed, observed, measured, stressed, broken on purpose, and explained with evidence.

It answers one core question with numbers, not opinions:

How much performance and resource overhead does observability add — and how much does it actually improve debugging and failure diagnosis?

This repository is a controlled experimental artifact. It does not claim general superiority of any observability stack, database, or orchestration platform. All conclusions are limited to the implemented workload, dataset, hardware, and experimental protocol — they characterize this system, this machine, and this telemetry implementation, and should not be generalized to all microservice systems. Stating these bounds is scientific honesty, not a limitation of the method.

Scope notes:

• RQ2 measures failure detection (MTTD), not full debuggability. Root-cause diagnosis requires a controlled operator study and is explicitly future work.
• Single machine / single stack (Apple M4, 16 GiB, Node.js/TypeScript). A different runtime, instrumentation library, or hardware budget could shift both the magnitude and the ordering of the costs.
• The numbers characterize the artifact; the methodology and qualitative ordering are the transferable contributions.

For reviewers / supervisors: docs/manuscript.md (paper draft) · docs/claims-to-evidence.md (every claim → command → data) · docs/demo.md (5-minute live walkthrough) · docs/ru/README.md (полная документация на русском).

flowchart LR
 k6([🧪 k6 / Client]) --> GW[API Gateway]
 GW --> TX[Transaction Service]
 TX --> PG[(🐘 PostgreSQL)]
 TX --> RD[(⚡ Redis)]
 TX --> PAY[Payment Service]
 TX --> MQ{{🐇 RabbitMQ}}
 MQ --> WK[Worker Service]
 WK --> PG
 subgraph OBS [🔭 Observability]
 OT[OTel Collector] --> PR[Prometheus]
 OT --> LO[Loki]
 OT --> JA[Jaeger]
 PR --> GR[📊 Grafana]
 LO --> GR
 JA --> GR
 end
 GW -. OTLP .-> OT
 TX -. OTLP .-> OT
 PAY -. OTLP .-> OT
 WK -. OTLP .-> OT

The request path: POST /transactions → API Gateway → Transaction Service → PostgreSQL write → Redis cache → Payment Service → RabbitMQ publish → Worker consume → event persisted. One flow that crosses HTTP, SQL, cache, and async messaging — enough surface area to measure something real.

Flip the entire telemetry depth with one environment variable. Every layer cleanly degrades to a no-op when disabled.

The realistic load campaign (pnpm load:run, open-model, N=10 per mode) feeds the statistics pipeline (STATS_DATASET=load pnpm stats:report) for non-parametric analysis with bootstrap CIs. The headline RQ1 result:

💡 Takeaway: metrics are essentially free (not statistically distinguishable from baseline), structured logging is the dominant cost (+164% CPU, +177% p50, with severe tail spikes), and the batched OpenTelemetry pipeline stays smooth despite carrying the most telemetry — all backed by N=10, bootstrap CIs, Kruskal–Wallis, Mann–Whitney U, and Cliff's δ.

📄 Read the full write-up: the journal manuscript draft is docs/manuscript.md; the engineering report is docs/final-report.md (§6.0 = primary result), with all tables and box plots in docs/statistics-load-report.md and the literature review in docs/related-work.md.

Six faults, all off by default, toggled by environment variables — paired with a manual debugging protocol that measures time-to-detect and time-to-root-cause across observability modes.

➡️ Protocol & schema: docs/failure-injection-protocol.md · Generate the report: pnpm failure:report

Objective detection (pnpm mttd:run): rather than a subjective timing, MTTD is measured as the time from fault onset to Prometheus alert firing. A real result — the fault is severe but invisible without metrics:

🔬 A step change, not a gradient: observability converts an undetectable fault into one detected within ~one scrape interval. See docs/mttd-report.md.

pnpm indexing:run seeds 1,000,000 transactions (research-grade) across 100k users and captures real EXPLAIN (ANALYZE, BUFFERS) plans across 7 index strategies ×ばつ 5 query patterns (35 combinations), with bootstrap 95% CIs on the p95 query time and read-improvement vs write-penalty reported separately. A real result from this repo:

💡 Takeaway: indexes matching the query's leading columns turn 1M-row sequential scans into ~16-row index lookups, but every index adds +19% (partial) to +322% (3-column) write latency — the read-vs-write trade-off, measured at scale with bootstrap 95% CIs. Full tables, charts, and raw plans: docs/postgres-indexing-report.md.

The same experiment runs on MongoDB (pnpm indexing:mongo, 6 strategies ×ばつ 4 queries via explain("executionStats")), and pnpm sql-nosql:report produces a careful SQL-vs-NoSQL comparison that leads with the structural metric (rows/documents examined) — the apples-to-apples signal — with latency treated as indicative only (both engines at 1M rows/documents):

🔬 Both engines reduce work along the same structural lines. Per the project's anti-goals, no "X is faster than Y" claim is made — conclusions are scoped to this dataset, access pattern, and hardware. See docs/mongodb-indexing-report.md and docs/sql-nosql-comparison.md.

pnpm orchestration:run deploys the same core stack on Docker Compose and Docker Swarm, measuring startup, scaling, recovery, and resource overhead live. The Kubernetes manifests are authored and validated (kubeconform, 19/19 resources) but not run here (no local cluster). A real result:

💡 Takeaway: Compose is the simplest to start but is not an orchestrator — it can't scale a host-port-published service and won't restart a killed container. Swarm adds a small deploy block and gets the routing mesh + self-healing. Kubernetes offers the strongest primitives for the most configuration. Full tables and charts: docs/orchestration-comparison.md.

1 The Compose file includes the observability profiles (superset); the fair core-only comparison is Swarm (123) vs Kubernetes (356). 2 Kubernetes is authored + statically validated, not run in this environment.

# 1. Install & verify
pnpm install
pnpm test # 42 unit tests
pnpm typecheck
pnpm lint
pnpm build
# 2. Start the base stack (Postgres, Redis, RabbitMQ + services)
docker compose -f infra/docker/compose/docker-compose.base.yml up --build -d
pnpm migrate:postgres
pnpm seed:postgres
# 3. Create a transaction
curl -X POST http://localhost:3000/transactions \
 -H "content-type: application/json" \
 -d '{"userId":"user-1","amount":42,"currency":"USD","description":"Demo"}'

Run services locally in watch mode instead: pnpm dev

Each observability mode runs the same k6 scenario ×ばつ, samples Docker stats, captures telemetry volume, and writes raw + processed + report artifacts.

# Phase 3 — Baseline (no observability)
OBS_MODE=none pnpm baseline:run
# Phase 4 — Metrics
OBS_MODE=metrics docker compose -f infra/docker/compose/docker-compose.base.yml --profile metrics up --build -d
pnpm migrate:postgres && pnpm metrics:run
# Phase 5 — Metrics + Logs
OBS_MODE=metrics_logs docker compose -f infra/docker/compose/docker-compose.base.yml --profile metrics --profile logs up --build -d
pnpm migrate:postgres && pnpm metrics-logs:run
# Phase 6 — Metrics + Logs + Traces
OBS_MODE=metrics_logs_traces docker compose -f infra/docker/compose/docker-compose.base.yml --profile metrics --profile logs --profile traces up --build -d
pnpm migrate:postgres && pnpm metrics-logs-traces:run
# Phase 7 — Full OpenTelemetry pipeline
OBS_MODE=otel_full OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 \
 docker compose -f infra/docker/compose/docker-compose.base.yml --profile metrics --profile logs --profile traces --profile otel up --build -d
pnpm migrate:postgres && pnpm otel-full:run
# Phase 8 — Aggregate all modes into one comparison (no containers needed)
pnpm overhead:report

# Load profiles — drive the same flow at different shapes
k6 run load-tests/k6/smoke.js
k6 run load-tests/k6/stress.js # 50→100→200→500 VUs
k6 run load-tests/k6/spike.js # 10→300→10 VUs
k6 run -e VUS=100 -e DURATION=60m load-tests/k6/soak.js
# Failure scenarios — start the stack with a fault flag, then drive load
k6 run load-tests/k6/failure-payment-slow.js
k6 run load-tests/k6/failure-payment-errors.js
k6 run load-tests/k6/failure-db-slow.js
k6 run load-tests/k6/failure-rabbitmq-consumer.js
k6 run load-tests/k6/failure-redis.js
pnpm failure:report # detection / root-cause report + charts

Dashboards when the stack is up: Grafana :3004 · Prometheus :9090 · Jaeger :16686 · RabbitMQ :15674

results/
├── raw/ # k6 summaries, Docker stats, telemetry-volume JSON per run
├── processed/ # comparison CSVs (observability-overhead, debuggability)
└── charts/ # dependency-free SVG charts (latency, CPU, memory, overhead, volumes)
docs/
├── observability-overhead-report.md # Phase 8 cross-mode analysis
├── failure-injection-report.md # Phase 9 debuggability report
└── failure-injection-protocol.md # manual measurement protocol

v1.0 — Observability Laboratory (current) covers the full measurement story end-to-end:

Post-v1 research (in progress):

🔭 Optional extensions: running the Kubernetes manifests on a local cluster, and per-target k6 load tests.

Backend NestJS · TypeScript (strict) · pnpm workspaces Data PostgreSQL · MongoDB · Redis · RabbitMQ Observability OpenTelemetry · Prometheus · Grafana · Loki · Jaeger Load & Orchestration k6 · Docker Compose

Every figure, table, and dataset is regenerable from source. The exact environment (hardware, runtimes, image digests) is recorded by pnpm env:capture into results/environment.json, and the full reproduction guide — prerequisites, a command for every result, determinism/seeds, and a data-availability statement — is in docs/reproducibility.md.

To cite this work, use CITATION.cff (GitHub's "Cite this repository" button). A versioned, DOI-archived snapshot is on Zenodo: 10.5281/zenodo.20561281 .

• Code — MIT.
• Experimental data & figures (results/ and the generated reports) — CC-BY-4.0.