Architecture Decision Record

Design: fix(harness): ruflo memory calls leak child Node processes

Context

Component Diagram

┌─────────────────────────────────────────────────────────────────┐
│ ruflo_with_timeout() │
│ (wrapper function responsible for timeout + cleanup) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Process Group Manager (setsid isolation) │ │
│ │ ├─ Creates isolated process group at spawn (line 404) │ │
│ │ └─ All descendants inherit group membership │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ │
│ ├─ spawns ─────────────────────────┐ │
│ │ │
│ ┌──────────────────────────────────┐ │ │
│ │ Shell Subshell (bg_pid) │ │ │
│ │ • Creates new process group │ │ │
│ │ via setsid │ │ │
│ └──────────────────────────────────┘ │ │
│ │ │ │
│ ├─ execs ─────────────┐ │ │
│ │ │ │
│ ┌──────────────────────────────┐│ │ │
│ │ Ruflo/Node Process ││ │ │
│ │ • Same process group ││ │ │
│ │ • Parent = subshell ││ │ │
│ └──────────────────────────────┘│ │ │
│ │ │ │ │
│ ├─ spawns ──────────┐ │ │ │
│ │ │ │ │
│ ┌──────────────────────────────┐┌─┘ │ │ │
│ │ Grandchildren (agentdb, ││ │ │ │
│ │ LLM daemon, etc.) ││ │ │ │
│ │ • Same process group ││ │ │ │
│ │ • Parent = Node ││ │ │ │
│ └──────────────────────────────┘│ │ │ │
│ │ │ │ │
│ ┌──────────────────────────────────────────────────┴───┴──────┐│
│ │ Signal Delivery Layer ││
│ │ ├─ Timeout detected: kill -TERM -<negative_bg_pid> ││
│ │ │ (kills ENTIRE process group, not just direct children) ││
│ │ ├─ Grace period: sleep 1 ││
│ │ └─ Force kill: kill -KILL -<negative_bg_pid> ││
│ │ (ensures all descendants terminate) ││
│ └───────────────────────────────────────────────────────────┘│
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Resource Cleanup (guaranteed on all paths) │ │
│ │ ├─ rm -f "$_rft_tmp" (temp file) │ │
│ │ └─ wait $bg_pid (reap zombie) │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘

Interface Contracts

Public API: ruflo_with_timeout

// Runs a command with a wall-clock timeout and graceful process group termination.
// STDIN/STDOUT/STDERR are captured in a temp file and printed on success.
// On timeout or failure, circuit-breaker increments RUFLO_FAILURE_COUNT.
function ruflo_with_timeout(
 timeout_seconds: number, // Wall-clock limit (1–3600 seconds)
 ...command: string[] // Command + args to run (shell function or binary)
): {
 exit_code: number; // 0 = success, 124 = timeout, >0 = command failure
 stdout: string; // Output from command (printed to stdout)
 side_effects: {
 RUFLO_FAILURE_COUNT: number; // Incremented on failure, exported
 RUFLO_AVAILABLE: boolean; // May flip to false if failure_count >= 5
 };
}
// Error contracts:
// - Returns 1 (non-zero) immediately if ruflo is disabled (fail-open)
// - Catches mktemp failures → returns 1, doesn't run command
// - Catches timeout → sends SIGTERM then SIGKILL to process group
// - Catches wait failures → returns exit code from child
// - ALWAYS cleans up temp file, even on error

Internal Helper: _kill_process_group (NEW)

// Send a signal to an entire process group (all descendants of a parent PID).
// POSIX standard: negative PID = process group ID.
// Fallback to pkill when kill -s <sig> -<pid> fails.
function _kill_process_group(
 signal: string, // "TERM" or "KILL" (uppercase, no SIG prefix)
 pid: number // Process ID (will be negated to form group ID)
): {
 exit_code: number; // 0 = signal sent, 1 = signal failed/not supported
 fallback_used: boolean; // true if kill -s failed and pkill was used
}

Invariants

Data Flow

Success Path

1. ruflo_with_timeout(30, "shell_fn", "arg1")
 │
 ├─ mktemp → _rft_tmp created
 │
 ├─ ( setsid shell_fn "arg1" ) >_rft_tmp &
 │ └─ Subshell spawned, running in new process group
 │ ├─ Inherits setsid group isolation
 │ ├─ Spawns Node → inherited group
 │ └─ Node spawns grandchildren → inherited group
 │
 ├─ Poll: while kill -0 $bg_pid && waited < 30
 │ ├─ kill -0 checks process still alive (no signal sent)
 │ └─ sleep 1 per iteration
 │
 ├─ Process completes
 │ └─ wait $bg_pid → exit 0
 │
 ├─ cat _rft_tmp → print output
 │
 └─ rm -f _rft_tmp → cleanup, return 0

Timeout Path

1. Wait loop reaches timeout_s seconds
 │
 ├─ kill -0 $bg_pid → still alive after timeout
 │
 ├─ Send SIGTERM to entire process group:
 │ └─ kill -TERM -<negative_bg_pid> (negation sends to group)
 │
 ├─ Grace period: sleep 1
 │ ├─ Node gets SIGTERM, starts cleanup
 │ └─ Grandchildren get SIGTERM (same signal)
 │
 ├─ If still alive, send SIGKILL:
 │ └─ kill -KILL -<negative_bg_pid>
 │ ├─ Ungraceful termination
 │ └─ All descendants killed immediately
 │
 ├─ wait $bg_pid → reap zombie
 │
 ├─ rm -f _rft_tmp → cleanup
 │
 ├─ Increment RUFLO_FAILURE_COUNT (exported)
 │
 └─ return 124 (timeout exit code)

Fallback Path (no pkill/setsid)

1. If kill -TERM -<negative_pid> fails (not supported):
 │
 ├─ Call _kill_process_group with fallback:
 │ └─ pkill -TERM -P $bg_pid
 │ ├─ Kills direct children only (suboptimal)
 │ ├─ Grandchildren may survive
 │ └─ Better than nothing on systems without process group support
 │
 ├─ Emit warning: "Process group kill failed, using pkill fallback"
 │
 └─ Continue with grace period + SIGKILL

Error Boundaries

Error Propagation

Caller (e.g., ruflo_hive_init)
 │
 └─ ruflo_with_timeout(30, "some_cmd")
 │
 ├─ Any error inside → exit_code = 1 or 124
 │ └─ Caller receives non-zero exit
 │ └─ RUFLO_FAILURE_COUNT incremented (exported)
 │ └─ If counter ≥ 5, RUFLO_AVAILABLE set to false
 │
 └─ Caller decides: retry, skip, or disable ruflo
 (never aborts pipeline due to fail-open semantics)

Decision

Alternatives Considered

Implementation Plan

Phase 1: Core Fix

Phase 2: Test Coverage

Phase 3: Integration Validation

Validation Criteria

Risk Areas

Key Files