Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Pipeline Plan 20

Seth Ford edited this page Feb 13, 2026 · 4 revisions

Here's the implementation plan:

Implementation Plan: End-to-End Integration Test Suite in CI

Files to Create/Modify

File Action Purpose
scripts/sw-integration-test.sh Create Main integration test script (smoke + live tests)
templates/pipelines/integration.json Create Minimal pipeline template (intake → build → test, 3 iterations)
.github/workflows/integration-test.yml Create CI workflow: smoke on every PR, live tests with budget cap
package.json Modify Add test:integration and test:integration:live scripts
.github/workflows/test.yml Modify Add integration smoke tests as parallel job

Implementation Steps

Step 1: templates/pipelines/integration.json

Minimal 3-stage template: intake → build → test. Model: sonnet. Max 3 iterations. No audit/quality gates/PR stage.

Step 2: scripts/sw-integration-test.sh — Tier 1 Smoke Tests

Uses the existing test harness pattern (PASS/FAIL counters, ERR trap, colored output). Mock binaries but runs real pipeline orchestration:

  1. Dry-run smoke — verify --dry-run exits 0, outputs "Dry run", creates .claude/ dirs
  2. Stage ordering — mock pipeline runs intake → build → test in order, state file updated
  3. State file format — validate required fields, stage progress, timestamps
  4. Budget enforcement — 0ドル budget causes clean exit, not crash

Step 3: Tier 2 Live Tests (gated behind INTEGRATION_LIVE=true + ANTHROPIC_API_KEY)

  1. README change — real Claude modifies a README in a temp repo; verify git diff shows changes, clean status
  2. PR creation — full fast pipeline creates a real PR; verify PR exists and branch is clean
  3. Budget cap — live run stays under 1ドル.00; cost tracking confirms

Step 4: .github/workflows/integration-test.yml

  • smoke job: every PR, no secrets needed, runs smoke tests
  • live job: PRs with ANTHROPIC_API_KEY, budget=1ドル.00, 15min timeout
  • regression job: push to main, re-runs live tests post-merge
  • All jobs write per-stage results to $GITHUB_STEP_SUMMARY

Step 5: Update package.json with test:integration scripts

Step 6: Add smoke tests as parallel job in existing test.yml

Task Checklist (10 tasks)

  • Create templates/pipelines/integration.json
  • Create scripts/sw-integration-test.sh with Tier 1 smoke tests
  • Add Tier 2 live integration tests (gated behind env vars)
  • Create .github/workflows/integration-test.yml
  • Update package.json with new scripts
  • Update .github/workflows/test.yml with smoke job
  • Add CI summary reporting ($GITHUB_STEP_SUMMARY)
  • Run smoke tests locally — all pass
  • Verify npm run test:integration works
  • Update CLAUDE.md documentation

Definition of Done

  • npm run test:integration exits 0 (smoke tests, no API key needed)
  • npm run test:integration:live runs both tiers when API key present
  • CI runs smoke on every PR, live when secrets available, regression on merge
  • Budget hard-capped at 1ドル.00 per live run
  • Per-stage pass/fail in CI summary
  • Existing npm test still passes

Clone this wiki locally

AltStyle によって変換されたページ (->オリジナル) /