-
Notifications
You must be signed in to change notification settings - Fork 1
Pipeline Design 242
Root cause confirmed. Line 579 guards on "[" only — objects ({) skip Case 2 entirely, fall through to Case 3, which falsely blames jq. Now writing the ADR.
_extract_text_from_json() in scripts/sw-loop.sh:561-608 converts Claude's --output-format json responses into plain text for the build loop log. Claude can emit two JSON shapes:
-
Array:
[{"type":"result","result":"..."}]— the common case -
Object:
{"type":"result","result":"..."}— observed when Claude outputs a single result
The function has three case guards:
| Case | Guard (line) | Handles |
|---|---|---|
| 2 |
first_char == "[" AND command -v jq (L579) |
Arrays only |
| 3 |
first_char == "[" OR first_char == "{" (L599) |
"JSON but no jq" |
| 4 | everything else (L605) | Plain text passthrough |
The bug: When Claude emits a JSON object, first_char is {. Case 2 skips it (only checks [). Case 3 catches it and prints "JSON output but jq not available" — even though jq IS available. The object is copied raw (unparsed JSON) into the log instead of extracting the .result field.
Constraints:
- Bash 3.2 compatible (no associative arrays, no
${var,,}) -
sw-loop.shcannot be sourced in tests (no source guard;main()runs unconditionally) — tests extract the function viased - Existing tests cover: empty files, valid JSON arrays, plain text, nested arrays, binary — but no JSON object test
Extend Case 2 to handle both [ and { first characters, branching on array vs object for the jq expression. This is the minimal fix that addresses the root cause.
Claude JSON output
├─ first_char == "[" or "{" AND jq available → Case 2 (jq extraction)
│ ├─ "[": jq '.[-1].result // empty' (array → last element)
│ └─ "{": jq '.result // empty' (object → direct field)
│ ├─ found .result → write to log, return
│ ├─ try .content fallback (array: '.[].content', object: '.content')
│ └─ neither found → warn "no .result field", write placeholder
├─ first_char == "[" or "{" AND jq NOT available → Case 3 (raw copy)
│ └─ warn "jq not available" (now accurate)
├─ empty/missing file → Case 1 (placeholder or stderr)
└─ anything else → Case 4 (plain text passthrough)
- All jq calls use
2>/dev/null+|| trueto suppress parse errors on malformed JSON - If jq succeeds but
.resultis null/empty, falls back to.content, then to a placeholder with a warning pointing to the raw file - Case 3's warning message is now only reachable when
command -v jqgenuinely fails — the message becomes accurate by construction
The fix uses only [[ ]] string comparisons and || operators — no features beyond Bash 3.2.
-
Fix only the warning message in Case 3 (Option B from issue) — Pros: One-line change, zero risk of regression. Cons: Doesn't fix the actual problem — JSON objects would still be copied raw into the log instead of extracting
.result. Users still see unparsed JSON in build logs. Treats the symptom, not the cause. -
Normalize all JSON to arrays before Case 2 — Pros: Single jq codepath, no branching. Cons: Requires an extra jq invocation (or
sedhack) to wrap objects in[]. More complex, higher risk of breaking valid array inputs. Over-engineered for a two-branch problem. -
Use Python/Node for JSON parsing instead of jq — Pros: Richer parsing capabilities. Cons: Adds a runtime dependency to a function that runs in the hot path of every build loop iteration. jq is already a project dependency. Massive over-engineering.
- Files to create: None
-
Files to modify:
-
scripts/sw-loop.sh(lines 578-603) — Restructure Case 2 guard and add object jq expressions -
scripts/sw-loop-test.sh— Add 3 new test cases for JSON object extraction
-
- Dependencies: None (jq already required)
-
Risk areas:
- The
sed -n '/^_extract_text_from_json()/,/^}/p'extraction in tests matches the first}at column 0. The restructured function must keep its closing}as the first such line, or the test extraction will break. Verified: the fix adds interiorif/else/fiblocks but doesn't change the function boundary. - Existing Test 18 (
valid.jsonwith array input) is the primary regression canary — it must continue to pass unchanged.
- The
- JSON array input
[{"result":"Hello"}]still extracts "Hello" (regression guard — Test 18) - JSON object input
{"result":"Object works"}extracts "Object works" (new test) - JSON object with
.contentfallback{"content":"Fallback"}extracts "Fallback" (new test) - No "jq not available" warning emitted when jq IS available and input is a JSON object (new test)
- Plain text passthrough still works (regression guard — Test 19)
- Empty file handling still works (regression guard — Test 17)
- Binary input doesn't crash (regression guard — Test 21)
- All existing
sw-loop-test.shtests pass - No Bash 3.2 compatibility violations (
declare -A,readarray,${var,,}etc.)
-
Missing
{guard in Case 2 (95%) — Line 579 checksfirst_char == "["only. Evidence: reading the code directly confirms the guard. A JSON object withfirst_char == "{"skips this block entirely. -
Case 3 guard is too broad (4%) — Line 599 catches both
[and{, so objects that should have been handled by Case 2 get a false "no jq" warning. This is a consequence of #1, not an independent cause. -
jq actually unavailable in some environments (1%) — Ruled out: the
command -v jqcheck on line 579 is never reached for objects because the"["check fails first. The jq availability is irrelevant to the bug.
-
scripts/sw-loop.sh:579:if [[ "$first_char" == "[" ]] && command -v jq— confirmed array-only guard -
scripts/sw-loop.sh:599-600:if [[ "$first_char" == "[" || "$first_char" == "{" ]]thenwarn "JSON output but jq not available"— confirmed false warning path for objects -
scripts/sw-loop-test.sh:294-296: Test 18 only tests array JSON[{...}]— confirmed no object test exists - Plan artifacts: The implementation plan correctly identifies the root cause and proposes the right fix (extend Case 2)
Extend Case 2's guard from "[" to "[" || "{", then branch internally on array vs object for jq expressions. This fixes the root cause (jq never tried for objects) rather than the symptom (wrong warning message). No previous failed attempt exists for this issue — this is the first execution.
- Run existing
sw-loop-test.sh— all 21+ tests must pass unchanged - New test: JSON object with
.result→ extracts text correctly - New test: JSON object with
.content→ fallback extraction works - New test: capture stderr during object extraction → no "jq not available" warning
- Verify the
sedfunction extraction in tests still works with the restructured function