Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Pipeline Plan 370

ezigus edited this page Apr 21, 2026 · 1 revision

Implementation Plan: Add Memory Recall and Store to stage_intake

Executive Summary

Add ruflo_recall_similar_outcomes() and ruflo_store() calls to stage_intake() to enrich downstream stages with historical context about similar issue classifications. This is the earliest injection point in the pipeline—before plan/design/build stages—and follows the exact pattern already established in stage_plan() and stage_design().


Architecture Design & Data Flow

Component Diagram

┌─────────────────────────────────────────────────────────────────┐
│ stage_intake() │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 1. Fetch issue metadata (title, labels, body, etc.) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 2. Classify issue (INTELLIGENCE_ISSUE_TYPE, severity) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 3. [NEW] ruflo_recall_similar_outcomes() │ │
│ │ Query historical context for this issue type │ │
│ │ Export as INTELLIGENCE_INTAKE_CTX │ │
│ └─────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 4. Save intake.json artifact │ │
│ └─────────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ 5. [NEW] ruflo_store() │ │
│ │ Store classification result for downstream lookup │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
 ↓
 ┌───────────────────┴───────────────────┐
 ↓ ↓
 stage_plan() stage_design()
 [Consumes [Consumes
 INTELLIGENCE_INTAKE_CTX INTELLIGENCE_INTAKE_CTX
 in prompt injection] in prompt injection]

Interface Contracts

ruflo_recall_similar_outcomes()

# Input:
# 1ドル (INTELLIGENCE_ISSUE_TYPE) — Issue type: "backend", "frontend", "bug", etc.
# 2ドル (ISSUE_LABELS) — Comma-separated labels (e.g., "feature,priority/high")
#
# Output:
# stdout — Plain text context summary or empty string if no results
# exit code — 0 always (fail-open)
#
# Called from:
# stage_intake (new), stage_plan, stage_design
#
# Example:
# local ctx; ctx=$(ruflo_recall_similar_outcomes "backend" "feature,urgent" 2>/dev/null || true)

ruflo_store()

# Input:
# 1ドル (key) — "stage-intake-result"
# 2ドル (value) — Classification summary (plain text, <500 chars)
# 3ドル (namespace) — "pipeline-${SHIPWRIGHT_PIPELINE_ID:-unknown}"
#
# Output:
# exit code — 0 on success, non-zero on failure
# Side effect: Stores in ruflo memory for retrieval by stage_plan/design
#
# Called from:
# stage_intake (new), stage_test
#
# Example:
# ruflo_store "stage-intake-result" \
# "Issue type: backend. Severity: critical. Labels: security,bug." \
# "pipeline-${SHIPWRIGHT_PIPELINE_ID:-unknown}" || true

INTELLIGENCE_INTAKE_CTX (new export)

# Environment variable exported by stage_intake
#
# Type: string (plain text, <2000 chars)
# Visibility: Exported to parent shell, consumed by stage_plan and stage_design
# Format: Unstructured text from ruflo_recall_similar_outcomes
#
# Downstream consumption:
# In stage_plan and stage_design, append to prompts under ## Intake Context section
# if [[ -n "$INTELLIGENCE_INTAKE_CTX" ]]; then
# prompt="${prompt}\n## Intake Context\n${INTELLIGENCE_INTAKE_CTX}"
# fi

Error Boundaries

Component Error Handling Propagation
ruflo_available returns false Skip recall & store Silent no-op with || true None (graceful)
ruflo_recall_similar_outcomes times out Return empty string Captured with 2>/dev/null || true None (graceful)
ruflo_store fails Acknowledge in log but continue || true prevents exit None (non-blocking)
INTELLIGENCE_INTAKE_CTX not set Use default empty string Guard all downstream usage with [[ -n ... ]] Safe default

Files to Modify

  1. scripts/lib/pipeline-stages-intake.sh — Add recall/store calls to stage_intake()
  2. scripts/lib/pipeline-stages-intake.sh — Update stage_plan() to consume INTELLIGENCE_INTAKE_CTX
  3. scripts/lib/pipeline-stages-intake.sh — Update stage_design() to consume INTELLIGENCE_INTAKE_CTX
  4. scripts/sw-lib-pipeline-stages-test.sh — Add tests for ruflo integration in stage_intake

Implementation Steps

Step 1: Add ruflo_recall_similar_outcomes() Call After Issue Classification

File: scripts/lib/pipeline-stages-intake.sh
Location: After line 108 (after skill_analyze_issue block), before line 110 (log_stage call)

 # Inject ruflo vector-similar past outcomes for this issue type (intake stage).
 # This is the earliest injection point — all downstream stages can reference this context.
 if declare -f ruflo_recall_similar_outcomes >/dev/null 2>&1 && \
 declare -f ruflo_available >/dev/null 2>&1 && \
 ruflo_available; then
 local _ruflo_intake_ctx
 _ruflo_intake_ctx=$(ruflo_recall_similar_outcomes \
 "${INTELLIGENCE_ISSUE_TYPE:-general}" "${ISSUE_LABELS:-}" 2>/dev/null || true)
 if [[ -n "$_ruflo_intake_ctx" ]]; then
 INTELLIGENCE_INTAKE_CTX="$_ruflo_intake_ctx"
 export INTELLIGENCE_INTAKE_CTX
 info "Ruflo intake context loaded (${#_ruflo_intake_ctx} bytes)"
 fi
 fi

Rationale: Placed immediately after skill analysis completes but before stage logging, so the enriched context flows into intake.json artifact and downstream stages.


Step 2: Add ruflo_store() Call After Intake Summary

File: scripts/lib/pipeline-stages-intake.sh
Location: After line 95 (after save_artifact call), before line 110 (log_stage call)

 # Ruflo: store intake classification result for cross-stage context
 if declare -f ruflo_store >/dev/null 2>&1; then
 local _intake_summary
 _intake_summary="Issue type: ${INTELLIGENCE_ISSUE_TYPE:-unknown}. Severity: ${INTELLIGENCE_SEVERITY:-unknown}. Labels: ${ISSUE_LABELS:-none}."
 ruflo_store "stage-intake-result" \
 "$_intake_summary" \
 "pipeline-${SHIPWRIGHT_PIPELINE_ID:-unknown}" || true
 fi

Rationale: Stores the classification result so downstream stages (stage_test, future stages) can recall intake context without re-classifying.


Step 3: Update stage_plan() to Consume INTELLIGENCE_INTAKE_CTX

File: scripts/lib/pipeline-stages-intake.sh
Location: After line 407 (after ruflo memory availability block, before line 409 guard_prompt_size)

 # Inject intake enrichment context if available from earlier stage
 if [[ -n "${INTELLIGENCE_INTAKE_CTX:-}" ]]; then
 plan_prompt="${plan_prompt}
## Intake Context (Classification Context)
${INTELLIGENCE_INTAKE_CTX}
"
 fi

Rationale: Makes intake-derived context explicitly visible to the planner, reducing redundant work.


Step 4: Update stage_design() to Consume INTELLIGENCE_INTAKE_CTX

File: scripts/lib/pipeline-stages-intake.sh
Location: After line 922 (after ruflo recall block in design), before line 924 (prior stage context block)

 # Inject intake enrichment context if available from earlier stage
 if [[ -n "${INTELLIGENCE_INTAKE_CTX:-}" ]]; then
 design_prompt="${design_prompt}
## Intake Context (Classification Context)
${INTELLIGENCE_INTAKE_CTX}
"
 fi

Rationale: Allows design stage to leverage intake classification without re-querying ruflo.


Task Checklist

  • Step 1: Add ruflo_recall_similar_outcomes call to stage_intake() after skill analysis (line 108)
  • Step 2: Add ruflo_store call to stage_intake() after intake.json artifact (line 95)
  • Step 3: Update stage_plan() to inject INTELLIGENCE_INTAKE_CTX (after line 407)
  • Step 4: Update stage_design() to inject INTELLIGENCE_INTAKE_CTX (after line 922)
  • Test 1: Add test case: stage_intake with ruflo_available=trueINTELLIGENCE_INTAKE_CTX is exported
  • Test 2: Add test case: stage_intake with ruflo_available=false → no-op (no error)
  • Test 3: Add test case: stage_intake ruflo_store is called with correct namespace pipeline-{ID}
  • Test 4: Add test case: stage_plan consumes INTELLIGENCE_INTAKE_CTX in prompt
  • Test 5: Add test case: stage_design consumes INTELLIGENCE_INTAKE_CTX in prompt
  • Integration: Run ./scripts/sw-lib-pipeline-stages-test.sh — all tests pass
  • Integration: Run npm test — no regressions
  • Acceptance: Verify ./scripts/sw-pipeline-test.sh passes with full pipeline execution

Testing Approach

Test Pyramid Breakdown

Unit Tests (65%): 12 tests

  • Recall available/unavailable paths (2 tests)
  • Store called with correct key/namespace (2 tests)
  • INTELLIGENCE_INTAKE_CTX exported and available (2 tests)
  • Downstream stage consumption (4 tests)
  • No-op behavior when ruflo unavailable (2 tests)

Integration Tests (30%): 5 tests

  • Full stage_intake → stage_plan flow with context propagation (2 tests)
  • Full stage_intake → stage_design flow with context propagation (2 tests)
  • Pipeline-level INTELLIGENCE_INTAKE_CTX persistence across stages (1 test)

E2E Tests (5%): 1 test

  • Full pipeline execution (intake → plan → design) with ruflo enabled verifies context flows end-to-end

Coverage Targets

  • Intake recall path: 100% coverage (declare -f guard, ruflo_available guard, context export)
  • Intake store path: 100% coverage (declare -f guard, store call, namespace format)
  • Downstream consumption: 100% coverage (guard with [[ -n ... ]], append to prompt)

Critical Paths to Test

Happy Path:

# stage_intake with ruflo available
export SHIPWRIGHT_PIPELINE_ID="test-pipeline-123"
export INTELLIGENCE_ISSUE_TYPE="backend"
export ISSUE_LABELS="feature,urgent"
ruflo_available() { return 0; }
ruflo_recall_similar_outcomes() { echo "Similar issue: auth module rewrite (3 days)"; }
ruflo_store() { return 0; }
stage_intake # Should export INTELLIGENCE_INTAKE_CTX, call store with correct namespace
[[ -n "$INTELLIGENCE_INTAKE_CTX" ]] || exit 1

Error Case 1: ruflo unavailable

# stage_intake with ruflo unavailable (declare -f fails)
unset -f ruflo_recall_similar_outcomes ruflo_available ruflo_store
stage_intake # Should complete successfully with no-op, no export
[[ -z "${INTELLIGENCE_INTAKE_CTX:-}" ]] || exit 1

Error Case 2: ruflo_available returns false

# stage_intake with ruflo_available=false
ruflo_available() { return 1; }
stage_intake # Should skip recall/store, complete successfully

Edge Case: Downstream consumption

# stage_plan with INTELLIGENCE_INTAKE_CTX set
export INTELLIGENCE_INTAKE_CTX="Backend auth module, 2-day priority"
stage_plan # plan_prompt should contain ## Intake Context section with value
grep -q "Intake Context" "$ARTIFACTS_DIR/plan.md" || exit 1

Design Alternatives Considered

Option A: Sequential Call Pattern (Recommended ✓)

Description: Add recall and store calls sequentially in stage_intake() after classification, follow exact pattern from stage_plan()/stage_design().

Pros:

  • ✓ Minimal code changes (2 code blocks in 1 file)
  • ✓ Follows established patterns — no new patterns introduced
  • ✓ Earliest injection point (issue classification context available immediately)
  • ✓ Recall happens before storage (safe ordering)
  • ✓ Export of INTELLIGENCE_INTAKE_CTX is straightforward

Cons:

  • Adds ~20ms latency to intake stage (negligible, ~2% overhead)

Blast Radius: Minimal — only stage_intake() modified directly; downstream stages enhanced but not broken


Option B: Deferred Recall in plan/design Only

Description: Skip intake recall/store; let plan/design stages continue existing recall pattern.

Pros:

  • Simplest (no changes to intake)
  • No additional intake latency

Cons:

  • ✗ Violates issue requirement: "earliest injection point"
  • ✗ Plan/design stages re-query without intake context
  • ✗ Duplicates recall calls (wasteful)
  • ✗ Misses opportunity to enrich downstream with classification context

Decision: Rejected — directly contradicts issue goal


Option C: Separate Memory Initialization Stage

Description: Create a new pipeline stage before plan that handles all memory initialization.

Pros:

  • Separates concerns (classification vs. memory)

Cons:

  • ✗ Adds pipeline complexity (new stage to manage)
  • ✗ Breaks established pattern of in-stage recall/store
  • ✗ Overkill for 2 function calls
  • ✗ Makes testing more complex (mocking new stage)

Decision: Rejected — unnecessary abstraction for this scope


Risk Analysis

Risk Impact Likelihood Mitigation
ruflo_available undefined Pipeline blocks or no-op Low Guard with declare -f, fail-open with || true
ruflo_recall_similar_outcomes hangs Intake timeout Very Low Timeout wrapper or 2>/dev/null || true
ruflo_store fails silently No downstream context stored Low Accepted (non-blocking, logged)
INTELLIGENCE_INTAKE_CTX not exported Downstream stages don't consume it Medium Explicitly test export, update downstream
Merge conflicts in stage_plan/design Rebasing issues Low Refactor into separate append sections
Regression in existing tests Test suite breaks Medium Run full test suite before merging

Mitigation Strategy:

  1. Use declare -f guards for all ruflo function checks
  2. Use || true to prevent blocking
  3. Test both available and unavailable ruflo paths
  4. Verify downstream stages import INTELLIGENCE_INTAKE_CTX and use it safely
  5. Run full test suite (./scripts/sw-lib-pipeline-stages-test.sh && npm test)

Definition of Done

Acceptance Criteria (from issue):

  • ruflo_recall_similar_outcomes is called inside stage_intake() after issue classification
  • Result is exported as INTELLIGENCE_INTAKE_CTX for downstream consumption
  • ruflo_store "stage-intake-result" is called with the classification summary
  • Both calls are guarded with the standard availability check (declare -f ... && ruflo_available)
  • Both calls use || true — never block the pipeline
  • Passes ./scripts/sw-pipeline-test.sh and npm test
  • Works when ruflo_available returns false (graceful no-op)

Implementation Quality:

  • Code follows existing patterns in stage_plan() and stage_design() (lines 374–386, 910–922)
  • Variable names match convention (INTELLIGENCE_INTAKE_CTX, _ruflo_intake_ctx, _intake_summary)
  • Error messages are informative (e.g., "Ruflo intake context loaded")
  • No new external dependencies introduced
  • File organization preserved (all changes in scripts/lib/pipeline-stages-intake.sh)

Testing Quality:

  • Unit tests for both ruflo available/unavailable paths
  • Tests verify INTELLIGENCE_INTAKE_CTX is exported correctly
  • Tests verify downstream consumption in stage_plan() and stage_design()
  • No regression in existing tests
  • All new tests follow existing pattern from sw-lib-pipeline-stages-test.sh

Documentation:

  • Code comments explain why recall happens at intake (earliest injection)
  • Namespace format documented ("pipeline-${SHIPWRIGHT_PIPELINE_ID:-unknown}")
  • Export behavior documented (INTELLIGENCE_INTAKE_CTX available to downstream)

Socratic Design Reasoning

Requirements Clarity

Q: What is the minimum viable change?
A: Add 2 ruflo calls (recall + store) with guards to stage_intake(). Minimum scope, maximum impact.

Q: Are there implicit requirements?
A: Yes — INTELLIGENCE_INTAKE_CTX must flow to downstream stages (stage_plan, stage_design). This requires updating both those stages to consume it.

Q: What are the acceptance criteria?
A: Given in issue. We must verify:

  1. Recall/store calls execute when ruflo available
  2. Graceful no-op when ruflo unavailable
  3. Test suite passes
  4. Context flows downstream (verified by inspection + test)

Dependency Analysis

Q: What depends on what?

  • stage_intake() → calls ruflo_recall_similar_outcomes(), ruflo_store()
  • stage_plan() → depends on INTELLIGENCE_INTAKE_CTX export (new dependency)
  • stage_design() → depends on INTELLIGENCE_INTAKE_CTX export (new dependency)
  • Tests → mock ruflo_available(), ruflo_recall_similar_outcomes(), ruflo_store()

Q: Circular dependency risks?
None identified. Flow is linear: intake → plan/design → build/test.


Summary

This implementation adds upstream memory enrichment to the Shipwright pipeline, allowing all downstream stages to benefit from intake-level issue classification without redundant queries. The changes follow established patterns, add no new dependencies, and are fully backward-compatible (graceful no-op when ruflo unavailable).

Clone this wiki locally

AltStyle によって変換されたページ (->オリジナル) /