feat(logging): add HTTP request/response interceptor - #2227

Closes #2224.

Some problems only show up in the raw traffic: a provider that rejects a header we thought we were sending, a proxy that rewrites a path, an integration that answers 200 with an error body. Neither the audit log (mutating, authenticated requests only) nor OpenTelemetry (spans, not wire data) shows that, so seeing it took a code change. Admin → Logging → HTTP Interceptor now captures it on both sides.

Built to the plan in the issue, with the open questions answered as agreed in this comment: logger.debug as the sink, no live tail, raw mode available, no SSE-body capture yet, global on/off, static assets and /api/health excluded.

What was added

Core — server/utils/httpInterceptor.js

• isInboundEnabled(req) / isOutboundEnabled(url) predicates, memoised on the identity of the raw config object (configCache hands back the same object until a reload), so the disabled path is an identity check plus a boolean read.
• recordInbound / recordOutbound emit through logger.debug under component HttpInterceptor, each stamped with the requestId from the per-request AsyncLocalStorage context — so an outbound provider call joins to the inbound /api/chat request that caused it:

  jq 'select(.component == "HttpInterceptor" and .requestId == "...")' logs/app.log

• interceptedFetch(fetchFn, ...) — one wrapper both outbound transports hand themselves to.
• Consolidated redactUrl / redactHeaders / redactBody, implementing the contract that server/utils/logRedactor.README.md described for a module which never existed. That orphaned doc is removed and the real functions are documented in docs/logging.md.

Capture points. Each outbound transport is split into a half that performs the request and a half that observes it, so the two concerns stay separate and the transport keeps the body it had on main:

Config — logging.http (schema in platformConfigSchema.js, defaults in server/defaults/config/platform.json, migration V084). Everything off by default. Reload is followed on every worker through configReloadHooks.

Admin UI — new section in AdminLoggingPage.jsx, saving through the existing PUT /api/admin/logging/config. It warns when capture is on while the log level is above debug, and surfaces the raw-mode warning only once bodies are enabled.

Guards

The risky parts of this feature are memory and privacy, so:

• Streams are never buffered. /api/chat, /api/inference, agent and workflow runs are recorded with status, headers and timing; their bodies are marked [STREAM]. Binary content types are skipped, and multipart/typed-array request bodies are named ([FORM-DATA], [BINARY n bytes]) rather than serialised.
• Outbound response bodies are peeked off a detached clone. clone() tees the body, and pipe stalls the source as soon as either branch fills — so awaiting a peek before returning the response would deadlock both. The peek runs detached with a timeout, and the record is written when it resolves.
• Bodies are capped (8 KB default) with the drop visible in the record: ...[TRUNCATED 8192 of 41003 bytes].
• Credentials are masked in URLs, headers and bodies. Header names, auth schemes, Set-Cookie attributes and LLM token counts (maxTokens, promptTokens, totalTokens) are deliberately kept — those are usually what you came to look at.
• Raw mode disables redaction and the cap, for the case where the redaction hides the value being chased. It warns accordingly.
• Auto-disable stops capture 60 minutes after it was switched on (configurable, 0 to disable), announced at info level. Enabling capture also writes one info line naming what was turned on, including when it was enabled by hand-editing platform.json — the one case the admin UI cannot warn about.

One fix that fell out of touching the outbound paths

Documented in docs/releases/5.5.0/fixes.md: PromptNodeExecutor's Google grounding-link resolution used raw fetch(), so it ignored the platform proxy/NO_PROXY/SSL configuration that every other outbound call honours. On deployments requiring an egress proxy the resolution silently failed.

I had also bundled in a safeFetch fix (a rejection from the undici request was treated as "undici unavailable" and silently retried through the fallback shim, surfacing a misleading error). That is no longer in this PR — it predates this branch and deserves its own review rather than riding along in a logging change. Happy to open it separately.

A note on the CodeQL alerts

Both js/request-forgery alerts that CodeQL raised on this PR were pre-existing sinks on main that the first version of this diff had relocated, so code scanning attributed them to the changed lines. They are resolved by the transport/observation split above, which leaves return nodeFetch(url, enhanced); and return await globalThis.fetch(url, { ...init, dispatcher }); as untouched context.

To be explicit: nothing is suppressed. Those alerts are unchanged on main and stay open on the branch. Worth knowing separately — this repository has no CodeQL workflow or config, so code scanning runs through GitHub's default setup, which does not honour inline // codeql[...] comments. The two such comments already in the tree (httpConfig.js:324, routes/admin/tools.js:1027) are therefore documentation for readers rather than working suppressions. See the review threads for the reasoning and a proposed follow-up audit of httpFetch's callers.

Verification

• server/tests/httpInterceptor.test.js — 34 checks, wired into npm run test:quick: redaction coverage (URL/header/body/cookie/XML/form), the maxTokens-is-not-a-token case, body cap and the truncation note, the >256 KB pattern-redaction path, maxBodyBytes: 0, raw mode end to end, allow/denylist selection for both directions, /api/health exclusion and path-segment prefix matching, auto-disable expiry and 0 = never, zero output when disabled, content-type gating, the inbound middleware against a real http.Server (bodies, SSE marked not buffered, static assets skipped), outbound success/failure/stream paths, and inbound↔outbound requestId correlation.
• Both transport splits are covered on the behaviours that moved with them: httpFetch still rejects a non-http(s) scheme and still applies the SSRF guard's pinned DNS lookup to the agent (without it, HttpNodeExecutor would silently lose its pinning); safeFetch still refuses a non-allow-listed private address and still throws on an unsupported protocol, and both refusals now appear in the wire log.
• npm run test:quick — 430 unit tests + all cluster/adapter suites pass. npm run test:integration:ci — 91 pass, 6 skipped.
• npm run lint:fix && npm run format:fix clean; no new ESLint findings (the one remaining AdminLoggingPage.jsx a11y warning is present at HEAD too).
• Server boots clean; V084 applies and is idempotent.
• Verified live against a running server: enabled capture through the admin API, confirmed /api/health excluded by the denylist, ?key= redacted in the recorded URL, password/apiKey masked in the request body, Cookie/Set-Cookie masked with attributes intact, and response bodies captured with timing. Sample record:

  {
   "component": "HttpInterceptor", "direction": "inbound",
   "requestId": "c32c469b-2753-4a80-95ec-24fba1c11c89",
   "method": "POST", "url": "/api/auth/local/login", "status": 200, "durationMs": 340.2,
   "requestHeaders": { "cookie": "authToken=[REDACTED]", "content-type": "application/json" },
   "responseHeaders": { "set-cookie": "authToken=[REDACTED]; Path=/; SameSite=Lax; HttpOnly" },
   "requestBody": "{\"username\":\"admin\",\"password\":\"pass...[REDACTED]\",\"apiKey\":\"sk-s...[REDACTED]\"}"
  }

• Verified the admin section in a browser (login → /admin/logging): the section renders with no console errors, all nine toggles work, the level warning appears while the level is above debug, and a save round-trips to platform.json with the comma-separated method and host lists parsed correctly (["post","put"], ["api.openai.com","*.anthropic.com"]).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KQRZBndxRcZhmh7WoKPEPe