Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Pipeline Plan 203

ezigus edited this page Mar 20, 2026 · 3 revisions

Plan stage complete. Here's the summary:

Approach: Source-based extraction (zero blast radius on callers)

3 new modules in scripts/lib/:

  • recruit-role-manager.sh (~900 lines): role definitions, matching, creation, evolution, self-tuning (12 functions)
  • recruit-team-composer.sh (~500 lines): team assembly, routing, specializations, decomposition (4 functions)
  • recruit-config-validator.sh (~900 lines): profiles, feedback, meta-learning, audit, stats (14 functions)

sw-recruit.sh shrinks to ~350 lines: shared state (paths, policy vars, utility functions) + sources the 3 modules + CLI router.

Key decisions:

  • Modules are sourced files, not standalone scripts — matches existing lib/ pattern
  • Source order: role-manager → team-composer → config-validator (respects dependency chain)
  • All 9 external callers (sw-pm.sh, sw-pipeline.sh, etc.) need zero changes — they invoke sw-recruit.sh as CLI
  • Section 18 test greps need updating to search lib/recruit-*.sh files

10 tasks, 3 new test files, full behavioral parity required before any new tests.

Full plan written to .claude/pipeline-artifacts/plan.md. volution, self-tuning (~900 lines) 2. lib/recruit-team-composer.sh -> team assembly, routing, specializations, decomposition (~500 lines) 3. lib/recruit-config-validator.sh -> agent profiles, feedback, meta-learning, audit, stats (~900 lines) 4. sw-recruit.sh -> thin orchestrator: shared state + source modules + CLI router (~350 lines) 5. All 80+ existing tests pass with zero behavioral changes 6. New modular tests for each extracted component

Design Alternatives

Approach A: Source-based extraction (CHOSEN)

  • Create lib/recruit-role-manager.sh, lib/recruit-team-composer.sh, lib/recruit-config-validator.sh
  • sw-recruit.sh sources all three and keeps the CLI router
  • External callers unchanged -- zero blast radius on integrations
  • Trade-offs: +simple, +safe, +incremental; -modules can still access each other's internals

Approach B: Standalone scripts with subprocess calls

  • Each module is an independent script invoked via bash
  • Trade-offs: +strong isolation; -breaks shared state, -requires serializing data between modules, -performance overhead, -major rewrite of internal data flow

Approach C: Single-file refactor with section markers

  • Keep one file, add clear section comments
  • Trade-offs: +zero risk; -doesn't achieve the goal of separate modules, -no independent testability

Decision: Approach A minimizes blast radius while achieving real modularity. Modules are sourced files (function libraries), not standalone scripts. This matches the existing lib/ pattern (e.g., lib/helpers.sh, lib/compat.sh, lib/pipeline-stages-build.sh).

Risk Assessment

Risk Impact Mitigation
Function ordering dependency Module B calls function from Module A before it's sourced Source in strict order: shared, role-manager, team-composer, config-validator
Shared variable scope Modules depend on ROLES_DB, PROFILES_DB, etc. Keep all storage paths and policy vars in orchestrator, sourced first
Test breakage from path changes Tests reference RECRUIT_SCRIPT directly No change -- tests still invoke sw-recruit.sh which sources modules
Grep-based static tests break Section 18 tests grep for functions in sw-recruit.sh Update greps to also search lib/recruit-*.sh files
Missing function in wrong module Subtle runtime error Each module has a guard variable; integration test verifies all commands work

Dependency Analysis

What depends on sw-recruit.sh (9 callers -- all use CLI interface):

  • sw-pm.sh -> team --json
  • sw-pipeline.sh -> ingest-pipeline, match --json (via lib/pipeline-stages-build.sh)
  • sw-triage.sh -> team --json
  • sw-swarm.sh -> match --json
  • sw-autonomous.sh -> match --json, team --json
  • sw-loop.sh -> team --json

None of these need changes -- they all invoke sw-recruit.sh as a CLI, and the CLI router stays in that file.

Internal dependencies between the three modules:

  • recruit-team-composer.sh calls: initialize_builtin_roles(), _recruit_keyword_match(), _recruit_has_claude(), _recruit_call_claude() -- from role-manager + shared
  • recruit-config-validator.sh calls: ensure_recruit_dir(), _recruit_locked_write(), initialize_builtin_roles(), _recruit_track_role_usage() -- from shared + role-manager
  • No circular dependencies exist

Architecture Decision Record

Component Diagram

+------------------------------------------------------------------+
| sw-recruit.sh (orchestrator) |
| Shared state (paths, policy vars, utility functions) |
| CLI router (case statement) |
| Sources: lib/compat.sh, lib/helpers.sh, sw-intelligence.sh |
| Sources: lib/recruit-role-manager.sh |
| Sources: lib/recruit-team-composer.sh |
| Sources: lib/recruit-config-validator.sh |
+--------+-----------------+-----------------+---------------------+
 | source | source | source
 v v v
+------------------+ +------------------+ +----------------------+
| recruit-role- | | recruit-team- | | recruit-config- |
| manager.sh | | composer.sh | | validator.sh |
| | | | | |
| Role defs | | cmd_team | | cmd_record_outcome |
| Matching | | cmd_route | | cmd_ingest_pipeline |
| cmd_roles | | cmd_specialize | | cmd_evaluate |
| cmd_match | | cmd_decompose | | cmd_profiles |
| cmd_create_role | | | | cmd_promote |
| cmd_evolve | | Depends on: | | cmd_onboard |
| cmd_invent | | role-manager | | cmd_mind |
| cmd_self_tune | | (matching fns) | | cmd_reflect |
| | | | | cmd_audit |
| No deps on | | | | cmd_stats |
| other modules | | | | cmd_help |
+------------------+ +------------------+ | |
 | Depends on: |
 | role-manager |
 | (role usage fns) |
 +----------------------+
External callers (unchanged):
 sw-pm.sh, sw-pipeline.sh, sw-triage.sh, sw-swarm.sh,
 sw-autonomous.sh, sw-loop.sh
 All invoke: bash sw-recruit.sh <cmd> [args]

Interface Contracts

// lib/recruit-role-manager.sh -- Role Assignment, Matching, Evolution
// Depends on: shared state (ROLES_DB, HEURISTICS_DB, MATCH_HISTORY, etc.)
initialize_builtin_roles(): void
cmd_roles(): void
cmd_create_role(args: string[]): void
_recruit_keyword_match(task: string): string
_recruit_llm_match(task: string, roles_json: string): string
_recruit_record_match(task, role, method, conf, agent_id): string
cmd_match(args: string[]): void
cmd_evolve(): void
_recruit_track_role_usage(role: string, outcome: string): void
cmd_invent(): void
cmd_self_tune(): void
_recruit_compute_population_stats(): void
// lib/recruit-team-composer.sh -- Team Assembly, Routing, Decomposition
// Depends on: role-manager (initialize_builtin_roles, _recruit_keyword_match)
cmd_team(args: string[]): void
cmd_route(args: string[]): void
cmd_specializations(): void
cmd_decompose(args: string[]): void
// lib/recruit-config-validator.sh -- Profiles, Feedback, Meta-Learning, Audit
// Depends on: role-manager (_recruit_track_role_usage, initialize_builtin_roles)
cmd_record_outcome(args: string[]): void
cmd_ingest_pipeline(args: string[]): void
cmd_evaluate(args: string[]): void
cmd_profiles(): void
cmd_promote(args: string[]): void
cmd_onboard(args: string[]): void
cmd_mind(args: string[]): void
_recruit_meta_learning_check(agent_id, outcome): void
cmd_reflect(): void
_recruit_reflect(): void
_recruit_meta_validate_self_tune(accuracy): void
cmd_audit(): void
cmd_stats(): void
cmd_help(): void

Data Flow

User/Script invokes: bash sw-recruit.sh <command> [args]
 |
 +------+------+
 | Orchestrator |
 | (sw-recruit) |
 | 1. Load env |
 | 2. Source |
 | modules |
 | 3. Route cmd |
 +------+------+
 +-------------+-------------+
 v v v
 role-manager team-composer config-validator
 | | |
 v v v
 ~/.shipwright/recruitment/ (shared JSON data stores)

Error Boundaries

  • Each cmd_* function handles its own argument validation and prints usage errors
  • LLM failures gracefully degrade to keyword/heuristic fallback (within role-manager and team-composer)
  • File I/O errors caught by set -euo pipefail + ERR trap (stays in orchestrator)
  • The orchestrator case statement handles unknown commands
  • No errors cross module boundaries -- each function is self-contained

Function-to-Module Mapping

sw-recruit.sh (orchestrator) -- keeps:

  • Lines 1-150: shebang, pipefail, SCRIPT_DIR, version, dependency check, compat/helpers source, fallbacks, _recruit_locked_write, storage paths, policy integration, policy vars, ensure_recruit_dir, intelligence source, _recruit_has_claude, _recruit_call_claude
  • Lines 2607-2645: Main CLI router
  • NEW: Three source statements for the extracted modules

lib/recruit-role-manager.sh -- gets:

  • initialize_builtin_roles (156-290)
  • _recruit_keyword_match (297-354)
  • _recruit_llm_match (357-386)
  • _recruit_record_match (390-431)
  • cmd_create_role (437-553)
  • _recruit_track_role_usage (746-763)
  • cmd_evolve (766-876)
  • _recruit_compute_population_stats (882-902)
  • cmd_self_tune (1755-1872)
  • cmd_roles (1878-1888)
  • cmd_match (1890-1999)
  • cmd_invent (1376-1488)

lib/recruit-team-composer.sh -- gets:

  • cmd_specializations (909-952)
  • cmd_route (955-1000)
  • cmd_team (1006-1181)
  • cmd_decompose (1654-1749)

lib/recruit-config-validator.sh -- gets:

  • cmd_record_outcome (560-680)
  • cmd_ingest_pipeline (683-740)
  • _recruit_meta_learning_check (1187-1236)
  • cmd_reflect (1239-1246)
  • _recruit_reflect (1248-1321)
  • _recruit_meta_validate_self_tune (1325-1370)
  • cmd_mind (1494-1648)
  • cmd_evaluate (2002-2069)
  • cmd_profiles (2071-2085)
  • cmd_promote (2087-2163)
  • cmd_onboard (2165-2259)
  • cmd_stats (2261-2316)
  • cmd_help (2318-2369)
  • cmd_audit (2383-2605)

Testing Strategy

Test Pyramid Breakdown

  • Existing tests (80+ in sw-recruit-test.sh): All pass unchanged -- behavioral parity proof
  • New unit tests (~30): 10 per module, testing functions in isolation
    • sw-role-manager-test.sh: role init, keyword matching, match recording, role creation, self-tune
    • sw-team-composer-test.sh: team heuristic composition, JSON output, route logic, decompose
    • sw-config-validator-test.sh: record-outcome, profile updates, ingest-pipeline, audit scoring
  • Integration tests (~5): Orchestrator sourcing all modules, cross-module flows

Coverage Targets

All cmd_* functions tested via existing tests (behavioral parity). New tests focus on recruit* internal functions.

Critical Paths to Test

  • Happy path: match --json returns valid JSON with primary_role, model, confidence
  • Error case: missing arguments produces usage error
  • Error case: LLM unavailable triggers keyword fallback with correct role
  • Edge case: empty roles DB triggers initialize_builtin_roles
  • Edge case: concurrent writes handled by _recruit_locked_write

Files to Modify

Action File
CREATE scripts/lib/recruit-role-manager.sh
CREATE scripts/lib/recruit-team-composer.sh
CREATE scripts/lib/recruit-config-validator.sh
MODIFY scripts/sw-recruit.sh
CREATE scripts/sw-role-manager-test.sh
CREATE scripts/sw-team-composer-test.sh
CREATE scripts/sw-config-validator-test.sh
MODIFY scripts/sw-recruit-test.sh

Implementation Steps

  1. Create lib/recruit-role-manager.sh: Extract role management functions with source guard (_RECRUIT_ROLE_MANAGER_LOADED)
  2. Create lib/recruit-team-composer.sh: Extract team composition functions with source guard (_RECRUIT_TEAM_COMPOSER_LOADED)
  3. Create lib/recruit-config-validator.sh: Extract validation/feedback functions with source guard (_RECRUIT_CONFIG_VALIDATOR_LOADED)
  4. Refactor sw-recruit.sh: Remove extracted functions, add three source statements after shared state initialization
  5. Update sw-recruit-test.sh: Fix Section 18 static grep tests to also search lib/recruit-*.sh files
  6. Run existing tests: Verify all 80+ tests pass (behavioral parity proof)
  7. Create sw-role-manager-test.sh: Unit tests for role-manager functions (source the module directly)
  8. Create sw-team-composer-test.sh: Unit tests for team-composer functions
  9. Create sw-config-validator-test.sh: Unit tests for config-validator functions
  10. Run full test suite: npm test to verify no regressions

Task Checklist

  • Task 1: Create scripts/lib/recruit-role-manager.sh with extracted role management functions
  • Task 2: Create scripts/lib/recruit-team-composer.sh with extracted team composition functions
  • Task 3: Create scripts/lib/recruit-config-validator.sh with extracted validation/feedback functions
  • Task 4: Refactor scripts/sw-recruit.sh into thin orchestrator
  • Task 5: Update scripts/sw-recruit-test.sh Section 18 static greps
  • Task 6: Run existing sw-recruit-test.sh -- verify all 80+ tests pass
  • Task 7: Create scripts/sw-role-manager-test.sh with ~10 unit tests
  • Task 8: Create scripts/sw-team-composer-test.sh with ~10 unit tests
  • Task 9: Create scripts/sw-config-validator-test.sh with ~10 unit tests
  • Task 10: Run npm test to verify full suite passes

Task Dependencies

  1. Tasks 1-3: Independent, can run in parallel
  2. Task 4: Blocks on Tasks 1-3
  3. Task 5: Blocks on Task 4
  4. Task 6: Blocks on Tasks 4-5
  5. Tasks 7-9: Block on Task 6 (verify parity before writing new tests)
  6. Task 10: Blocks on Tasks 7-9

Risk Analysis

Risk What could break Mitigation
Source order wrong Functions undefined at call time Source role-manager, team-composer, config-validator (in that order)
Shared variable not visible Module cant access ROLES_DB etc. All shared state in orchestrator before sourcing
Section 18 greps fail Tests grep for functions in sw-recruit.sh only Update greps to also search lib/recruit-*.sh
Double-source Functions redefined Guard variable in each module
set -euo pipefail in modules Conflicting error handling Dont add separate pipefail -- modules inherit from orchestrator

Alternatives Considered

Approach Complexity Blast Radius Achieves Goal
A: Source-based (chosen) Low Zero on callers Yes
B: Subprocess scripts High Medium Yes (stronger isolation)
C: Section comments only None Zero No

Definition of Done

  • Three new lib/recruit-*.sh files exist with extracted functions
  • sw-recruit.sh is < 400 lines (down from 2645)
  • All 80+ existing tests in sw-recruit-test.sh pass unchanged
  • New test files exist: sw-role-manager-test.sh, sw-team-composer-test.sh, sw-config-validator-test.sh
  • npm test passes with no regressions
  • No external caller changes required (sw-pm.sh, sw-pipeline.sh, etc.)
  • All modules have source guards to prevent double-loading
  • Bash 3.2 compatible (no associative arrays, no readarray, etc.)

Clone this wiki locally

AltStyle によって変換されたページ (->オリジナル) /