-
Notifications
You must be signed in to change notification settings - Fork 1
Pipeline Design 23
The pipeline artifacts directory is also restricted. Here's the complete ADR:
Shipwright's daemon (sw-daemon.sh) polls GitHub for issues every 30–60 seconds via daemon_poll_issues(). This introduces up to 60 seconds of latency between a user labeling an issue and the pipeline starting. GitHub supports webhook delivery for issues.labeled events, enabling sub-second notification.
Constraints:
- The dashboard server (
dashboard/server.ts, Bun) already serves HTTP at port 8767 with an existing/api/webhook/ciendpoint (line 4074) — this is the natural home for webhook reception. - The daemon is a bash process with no HTTP server. Communication between dashboard and daemon must use the filesystem (established pattern:
daemon-pause.flagatsw-daemon.sh:3272,daemon.shutdownat line 188,daemon-state.json). - All bash must be Bash 3.2 compatible (
set -euo pipefail, no associative arrays, noreadarray). - Atomic file writes required (write to tmp +
mv). - The daemon's poll loop already sleeps in 1-second intervals (
sw-daemon.sh:4197–4202), providing a natural hook point for trigger file detection.
The dashboard receives GitHub webhooks, validates HMAC-SHA256 signatures, and writes trigger files to ~/.shipwright/webhook-triggers/. The daemon detects these files during its 1-second sleep loop and runs an immediate poll cycle.
GitHub ──POST──▶ dashboard:8767/api/webhook/github
│
├─ Validate X-Hub-Signature-256 (HMAC-SHA256)
├─ Filter: only issues.labeled matching watch_label
├─ Write trigger: ~/.shipwright/webhook-triggers/<issue>.json
├─ Log delivery: ~/.shipwright/webhook-deliveries.jsonl
└─ Broadcast to WS clients
│
daemon poll loop ─┘
(every 1s sleep tick)
├─ Check ~/.shipwright/webhook-triggers/ for files
├─ Remove trigger files (consume)
└─ Call daemon_poll_issues (immediate cycle)
Uses Web Crypto API (built into Bun). Constant-time comparison via Bun's built-in crypto. No external dependencies. Reads X-Hub-Signature-256 header, computes HMAC-SHA256 of the raw request body with the configured secret, and rejects with 401 on mismatch.
{
"issue_number": 42,
"repo": "owner/repo",
"label": "ready-to-build",
"timestamp": "2026年02月11日T22:00:00Z",
"delivery_id": "abc-123"
}Atomic writes: write to .tmp then mv. One file per issue to deduplicate rapid re-labels.
New function daemon_check_webhook_triggers() called inside the 1-second sleep loop (between lines 4199–4201). Checks ~/.shipwright/webhook-triggers/ for JSON files, logs each trigger, removes the files, and calls daemon_poll_issues if any were found.
The existing sleep loop becomes:
local i=0 while [[ $i -lt $effective_interval ]] && [[ ! -f "$SHUTDOWN_FLAG" ]]; do sleep 1 daemon_check_webhook_triggers || true i=$((i + 1)) done
~0ms overhead per tick when no triggers exist (directory check + early return).
New daemon_init_webhook() function: generates secret via openssl rand -hex 32, writes to daemon-config.json under webhook.secret, creates GitHub webhook via gh api repos/{owner}/{repo}/hooks with issues event scope, writes webhook.dashboard_url to config.
{
"webhook": {
"secret": null,
"dashboard_url": null
}
}Loaded in load_config() (~line 362) as WEBHOOK_SECRET and WEBHOOK_DASHBOARD_URL. When secret is null, webhook endpoint returns 503. Polling continues regardless.
POST /api/webhook/github (public route — added to isPublicRoute() at server.ts:370):
- Validate
X-Hub-Signature-256against configured secret - Parse
X-GitHub-Eventheader; only processissuesevents withaction: "labeled" - Check label matches
watch_labelfromdaemon-config.json - Write trigger file + delivery log to
~/.shipwright/webhook-deliveries.jsonl - Return 200 on success, 401 on bad signature, 503 if not configured
- Idempotent: same issue number overwrites the trigger file
GET /api/webhook/status (protected route):
- Read
~/.shipwright/webhook-deliveries.jsonl - Return last N deliveries, total count, error count
Add webhook field to FleetState interface (server.ts:125):
webhook?: { configured: boolean; total_deliveries: number; recent_deliveries: number; last_delivery_at: string | null; errors_24h: number; };
Populated from webhook-deliveries.jsonl in getFleetState() (server.ts:697).
| Failure | Behavior |
|---|---|
| Invalid HMAC signature | 401 response, logged, no trigger file written |
| Dashboard down | GitHub retries; polling continues as fallback |
| Daemon not running | Trigger files accumulate, consumed on next start |
| Trigger dir missing | Created lazily by dashboard; daemon early-returns safely |
| Malformed trigger JSON | Daemon logs warning, removes file, continues |
| Disk full | Atomic write fails, 500 to GitHub, polling continues |
| Rapid re-labels | Same issue number overwrites trigger file (dedup) |
- Webhook is opt-in. Without
webhook.secret, endpoint returns 503. - Polling continues identically when webhook is not configured.
- No changes to existing behavior. Trigger check is a no-op when trigger directory doesn't exist.
-
Standalone webhook server (separate process) — Pros: Independent lifecycle, simpler code isolation / Cons: Another port to expose, another process to manage, duplicates HTTP infrastructure in
dashboard/server.ts, still requires filesystem IPC -
Named pipe / Unix socket between dashboard and daemon — Pros: Truly instant (~0ms vs ~1s), no filesystem polling / Cons: Complex lifecycle management, Bash 3.2 named pipe is fragile under
set -e, doesn't survive daemon restarts (trigger files do), harder to debug -
Direct daemon HTTP server (bash + netcat/socat) — Pros: No dashboard dependency / Cons: Extremely fragile, no TLS, no HTTP parsing, blocks main loop, security risk
-
GitHub Actions workflow as intermediary — Pros: No inbound network needed / Cons: 5–15s Actions startup latency, CI minutes cost, more moving parts
-
Files to create:
-
scripts/sw-webhook-test.sh— Test suite for webhook trigger handling,--webhookCLI flag, config loading
-
-
Files to modify:
-
dashboard/server.ts—POST /api/webhook/github,GET /api/webhook/status, HMAC validation, trigger file writing, delivery log,isPublicRoute(),FleetState.webhook,getFleetState() -
scripts/sw-daemon.sh—webhook.*inload_config()(~line 362) and defaults (~line 207),daemon_check_webhook_triggers(), sleep loop (~line 4199),daemon_init_webhook(),--webhookCLI flag (~line 256), config template (~line 4521), help text -
scripts/sw-daemon-test.sh— Webhook config loading + trigger file detection tests -
.claude/CLAUDE.md— Document webhook config and runtime state paths -
package.json— Registersw-webhook-test.sh
-
-
Dependencies: None. Bun's built-in Web Crypto API for HMAC. No new npm packages.
-
New runtime state:
-
~/.shipwright/webhook-triggers/<issue>.json— Trigger files (transient, consumed by daemon) -
~/.shipwright/webhook-deliveries.jsonl— Delivery log (append-only)
-
-
Risk areas:
- Race condition on trigger files — Mitigated by atomic writes (tmp + mv) and daemon consuming files after read
-
Trigger file accumulation when daemon stopped — On restart, all consumed at once but
daemon_poll_issues()respectsMAX_PARALLELand queues excess - HMAC timing attack — Constant-time comparison via Bun's built-in crypto
-
Dashboard config read — Dashboard reads
daemon-config.jsonforwatch_labelandwebhook.secret; same pattern as existingdaemon-state.jsonreads - Test isolation — Mock trigger directory and file operations; no real dashboard or GitHub
-
POST /api/webhook/githubreturns 401 for invalid HMAC signatures -
POST /api/webhook/githubreturns 503 when no webhook secret is configured -
POST /api/webhook/githubwrites trigger file only forissues.labeledevents matchingwatch_label -
POST /api/webhook/githubignores wrong label, wrong action, or wrong event type - Trigger files use atomic writes (tmp + mv)
-
daemon_check_webhook_triggers()processes and removes trigger files within 1 second -
daemon_check_webhook_triggers()is a no-op when trigger directory doesn't exist -
daemon_check_webhook_triggers()handles malformed JSON without crashing - Polling continues identically when webhook is not configured
-
daemon init --webhookgenerates secret and creates GitHub webhook viagh api -
GET /api/webhook/statusreturns delivery history with counts - All 22 existing test suites pass
- New
sw-webhook-test.shsuite passes - All new bash is Bash 3.2 compatible
- HMAC comparison is constant-time
The ADR is ready. I wasn't able to write it to .claude/pipeline-artifacts/design.md due to file permissions — you'll need to save it there or grant write access to that directory. The design follows established Shipwright patterns (filesystem IPC, atomic writes, error-guarded poll loop calls) and requires zero new dependencies.