Skip to content

Navigation Menu

Sign in
Sign up

feat(aarch64): RQ-64-MACHO — a Mach-O MH_OBJECT over the SAME ObjectPlan, linked by Apple ld and EXECUTED on macOS - #1184

Merged
avrabe merged 10 commits into
main from
feat/macho-1184
Sep 7, 2026
Merged

feat(aarch64): RQ-64-MACHO — a Mach-O MH_OBJECT over the SAME ObjectPlan, linked by Apple ld and EXECUTED on macOS #1184
avrabe merged 10 commits into
main from
feat/macho-1184

Conversation

@avrabe

@avrabe avrabe commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

RQ-64-MACHO (#242) — the container, not the ISA

-b aarch64 --relocatable already emitted a linkable SysV ELF; #1179 proved it
links and executes on arm64 Linux. This adds the macOS container over
identical instruction bytes
— no new ISA, no new proofs.

The structural move

The Mach-O writer does not re-derive symbol ordering or the #1013 rules.
Both containers render one ObjectPlan, so a second container disagreeing
with the first is unrepresentable, not merely tested against. That is the
same move VCR-ISA-001 (#667) made for the selector's Rocq model, and the
structural answer to the divergence class #1180 belongs to. The ELF corpus is
140/140 byte-identical across that refactor.

Oracle landed FIRST

The first commit is the oracle, before any writer existed:

  • byte-identity sweep (aarch64-oracle step): for every .wat the aarch64
    backend accepts, both containers are emitted and .text/.data, the symbols
    (under Darwin's _) and the relocation set must be identical, with the
    acceptance decision agreeing. Host-independent — no linker, no execution.
  • macho-host-link-native on macos-latest: asserts the runner really is
    arm64 macOS, links with Apple ld, EXECUTES against wasmtime-first values.
    REQUIRE_NATIVE=1 so a wrong host is RED, never a silent skip. Deliberately
    not a required check, per feat(aarch64): RQ-64-ARM64LINUX — the arm64-Linux host-library claim, linked by ld.lld and EXECUTED (natively, qemu-user, unicorn) vs wasmtime #1179 — a runner pool GitHub cannot schedule
    would deadlock every merge.

Provenance, stated rather than blurred

The lane that built this was terminated mid-step by an API session limit and the
coordinator finished it. The artifact's verified-by splits the claims: the
macOS 17/17 execution is the lane's measurement, for CI to confirm; the
rebase resolution, compile, YAML parse and gate results are the coordinator's,
re-run on the merged tree.

Cumulative counters RE-DERIVED, not carried

--min-oracles in aarch64-oracle is 17 on main because #1179 already moved
16→17, so this makes it 18 — copying the pre-rebase value would have
silently un-gated one oracle. The emulation floor is unchanged at 324847,
which is correct rather than an oversight: this differential declares
compiles >= 100, so the compiles bucket moves 9 scripts/43 → 10/143.
oracle_wiring: 190 scripts, 182 wired, 0 unwired debt.

One ratchet moved, with a waiver and a reason

SYNTH-SUBTRACTION-MIRRORS 59 → 60. Both new files contain "mirror" only to say
they are not one — elf.rs explains a second hand-written copy "would be
exactly the mirror the North Star forbids" (the reason ObjectPlan exists), and
the differential explains "a hand-mirrored pair would drift" (why it generates
the C harness and the wasmtime sequence from one case table). Rewording to dodge
the word count would delete the explanation and leave the metric happy.

Not claimed

Co-linking two synth Mach-O objects. #1180 records that the aarch64 ELF writer
emits every func_N as STB_GLOBAL; whether Mach-O shares that shape is
unmeasured and asserted neither way.

Refs #242

avrabe added a commit that referenced this pull request Sep 7, 2026
...s landed: (R10 attribution)
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
avrabe added a commit that referenced this pull request Sep 7, 2026
...s landed: (R10 attribution)
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
avrabe added a commit that referenced this pull request Sep 7, 2026
...s landed: (R10 attribution)
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
avrabe added a commit that referenced this pull request Sep 7, 2026
...s landed: (R10 attribution)
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.15126% with 11 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/synth-cli/src/main.rs 87.30% 8 Missing ⚠️
crates/synth-backend-aarch64/src/macho.rs 99.55% 2 Missing ⚠️
crates/synth-backend-aarch64/src/elf.rs 98.75% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

avrabe and others added 10 commits September 7, 2026 14:56
...dentity sweep + macOS host-link execution differential, before any writer exists
The oracle, then the writer — the gating rule this artifact was deferred
from v0.63 to honour. `scripts/repro/macho_host_link_rq64_differential.py`
follows #1179's shape (one case table generating both the C harness and
the wasmtime sequence, wasmtime-first, red-first controls built in) and
adds the increment this artifact is actually about:
 * BYTE IDENTITY on every host: `.text` == `__TEXT,__text`, `.data` ==
 `__DATA,__data`, symbols map 1:1 under the Darwin `_` prefix, and the
 relocation SET maps 1:1 under CALL26/JUMP26 -> BRANCH26, PG_HI21 ->
 PAGE21, LO12 -> PAGEOFF12 — for the fixture AND every repo `.wat` the
 backend accepts (139 today), with acceptance required to AGREE across
 containers. The Mach-O is read by the file's own ~60-line parser so the
 leg runs on the Linux runner too.
 * an identity MUTATION control (bit 30 of `add`'s first word flipped in
 `__text` must be reported);
 * on arm64-Darwin (REQUIRE_NATIVE=1 makes any other host RED, never a
 skip): clang -arch arm64 -ffixed-x28 + an x28 shim, linked by Apple ld
 through the clang driver, checked by Apple nm, EXECUTED natively vs
 wasmtime; the ELF twin handed to Apple ld must be refused `unknown
 file type` (measured), a missing `_host_add` refused by name, ld.lld
 must refuse the Mach-O when present, the mutated object linked and run
 must mismatch, and the #1180 `_func_1` co-link collision is pinned.
Measured RED against the flagless binary: exit 1, `unexpected argument
'--object-format'`, driver floor `compiles >= 100` unmet (measured 2).
The fixture is byte-for-byte #1179's module below its header (verified by
diff), so the same bytes' values on arm64-Linux and macOS read side by
side.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
..., so a second container cannot disagree with it (140/140 objects byte-identical)
`plan_object` now computes everything a relocatable object SAYS — the
concatenated `.text`, the `.data` image, the symbol list in emission order
(function aliases, `.data` symbols, the #1017 referenced externals; first
occurrence wins on a duplicated name), and the resolved relocations with the
#1013 unplaced-symbol refusal, message verbatim — and
`build_relocatable_object_full` only lays ELF around it. The Mach-O writer
that follows consumes the SAME plan, which is what makes `.text` identity
between the two containers a property of one function rather than of two
writers happening to agree (the North Star's "derive what you check against
from the artifact you ship"; a hand-copied symbol-ordering rule in a second
writer would be the mirror it forbids).
Gated as a pure refactor: every one of the 140 repo `.wat` modules the
aarch64 backend accepts (`scripts/repro/*.wat` + `tests/**/*.wat`) compiles
to bytes IDENTICAL to the pre-refactor binary (corpus captured before,
compared after: accepted=140 identical=140 differ=0), and the crate's 156
unit tests — including the layout-pinning ELF tests — pass unchanged. The
`let info = if f.is_object { 0x11 } else { 0x12 };` line #1179's claim pin
counts is kept verbatim.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
..._ARM64 MH_OBJECT over the ELF plan; Apple ld links it and macOS EXECUTES it 17/17 vs wasmtime
The container was the blocker, not the ISA — now measured rather than
argued. `crates/synth-backend-aarch64/src/macho.rs` lays Mach-O around the
SAME `ObjectPlan` the ELF writer consumes (the previous commit): one
unnamed LC_SEGMENT_64 with `__TEXT,__text` (+ `__DATA,__data` when the
module has globals, file offsets congruent with addresses),
LC_BUILD_VERSION (macOS, minos 11.0 — the one macOS-specific fact),
LC_SYMTAB + LC_DYSYMTAB with every symbol N_EXT under Darwin's `_` prefix,
and one `__text` relocation per planned relocation, all r_extern/r_length
2: CALL26/JUMP26 -> ARM64_RELOC_BRANCH26, ADR_PREL_PG_HI21 -> PAGE21,
ADD_ABS_LO12_NC -> PAGEOFF12 (addends are zero in both containers, so no
ADDEND entry is ever needed; any other kind fails loudly, the ELF rule).
MH_SUBSECTIONS_VIA_SYMBOLS is deliberately NOT set so `__text` stays one
atom with exactly the ELF layout.
The flag is EXPLICIT and narrow: `--object-format {elf,macho}`, default
`elf` (every pre-v0.64 compile byte-identical — the ELF path was
corpus-diffed 140/140 in the previous commit), and `macho` is REFUSED on
any backend but `-b aarch64` rather than silently emitting ELF under a flag
that says otherwise. `build_aarch64_object` in synth-cli is the one place
the container is chosen; both the single-function and `--all-exports`
paths route through it. The post-compile hint names the toolchain the
oracle actually exercised.
MEASURED, by the red-first oracle from two commits ago, now green
(`SYNTH=... python scripts/oracle_run.py
scripts/repro/macho_host_link_rq64_differential.py`, arm64 macOS 26.6,
Apple ld-1267, clang 21):
 * byte identity: `.text == __text` (580 B), `.data == __data`, 29 symbols
 and 20 relocations map 1:1 for the fixture; 139/139 corpus modules
 byte-identical, 0 acceptance disagreements (140 in all);
 * identity mutation control: a flipped bit in `__text` is reported;
 * Apple ld links the object with NO warnings; nm sees all 32 wanted
 symbols, `_host_add` bound to the C definition;
 * EXECUTED natively: 17/17 values match wasmtime (w/x/d registers,
 x28 memory from both sides, globals persisting in __DATA, call_indirect
 via BRANCH26 trampolines, import call, RQ-63-A64STACK shape);
 * refusals by name: the ELF twin into Apple ld (`unknown file type`), a
 missing `_host_add`, the Mach-O into ld.lld (`unknown file type`);
 * execution mutation control: the mutated object linked and run
 mismatches (add -> 0xffffffff vs 0x7);
 * #1180 pinned: two Mach-O objects collide `duplicate symbol '_func_1'`
 exactly as ELF does — noted, not fixed here;
 * driver floor compiles >= 100, measured 387.
Six unit tests in macho.rs pin the header/sections/build-version, the
symbol prefix + defined-then-undefined partition, the kind-for-kind reloc
packing, `.text`/`.data` identity with the ELF writer, the no-globals
single-section shape, and the shared #1013 refusal.
Refs #242, #1180
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
...ow, is_multiple_of, test type alias); ELF corpus still 140/140 identical
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
...ative arm64-macOS host-link oracle, so the container claim is GATED
Re-applied against main's ci.yml after #1179 landed. The first attempt was
dropped in the rebase: git's conflict boundary fell THROUGH a cache step rather
than between jobs, so a mechanical "keep both sides" produced invalid YAML with
an orphaned `key:` line. Re-applied surgically instead — the new step, the
ledger bump and the new job placed individually, then the file parsed (59 jobs)
rather than assumed correct.
 - byte-identity sweep, as a step in `aarch64-oracle`: for the fixture and
 every `.wat` the aarch64 backend accepts, BOTH containers are emitted and
 `.text`/`.data`, the symbols (under Darwin's `_`) and the relocation SET
 must be identical, with the acceptance decision agreeing. This is the
 host-independent half of the artifact's thesis — that the container, not
 the ISA, is what blocks shipping synth output as a host library — and it
 needs no linker and no execution to prove it.
 - `macho-host-link-native` on `macos-latest`: asserts the runner really is
 arm64 macOS, links with Apple ld and EXECUTES against wasmtime-first
 values. REQUIRE_NATIVE=1, so a wrong host is RED, never a silent skip.
 Not a required check, per the #1179 precedent: a new runner pool that
 GitHub cannot schedule would deadlock every merge.
CUMULATIVE COUNTERS RE-DERIVED ON THE MERGED TREE, never carried textually:
`--min-oracles` in aarch64-oracle is 17 on main because #1179 already bumped
16->17, so this makes it 18 — copying the pre-rebase value would have silently
un-gated one oracle. The emulation floor is UNCHANGED at 324847 and that is
correct, not an oversight: the Mach-O differential declares `compiles >= 100`,
so the compiles bucket moves 9 scripts/43 to 10/143 while emulations does not.
oracle_wiring reports 190 scripts, 182 wired, 0 unwired debt.
SYNTH-SUBTRACTION-MIRRORS: 59 -> 60 with a waiver, NOT a loosened gate. Both
new files say "mirror" only to state that they are not one — elf.rs explains
that a second hand-written copy of the symbol-ordering rules "would be exactly
the mirror the North Star forbids" (the reason ObjectPlan exists), and the
differential explains that "a hand-mirrored pair would drift" (why it generates
the C harness and the wasmtime sequence from one case table). Rewording to dodge
the word count would delete the explanation and leave the metric happy.
Verified: claim_check 60/60, oracle_wiring exit 0 at the pinned floor,
status_evidence exit 0, ci.yml parses, rivet at main's 40-error baseline.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
...ance split between lane and coordinator
The lane was terminated mid-step by an API session limit; the coordinator
finished the rebase and the CI wiring. The verified-by says which claims come
from which, and marks the macOS 17/17 execution as the lane's measurement for
CI to confirm rather than restating it as though re-run here.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
...s landed: (R10 attribution)
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
...ed a path that never exists
The `Mach-O host-link oracle, NATIVE arm64-macOS` job went RED at its FIRST
assertion, before running the oracle at all. The defect is in the wiring I added
in this PR, not in the lane's differential.
 "$(brew --prefix llvm)/bin/ld.lld" --version
Two things wrong at once, and each hides the other:
 - `brew --prefix llvm` prints a path whether or not the formula is installed,
 so the expression always LOOKS resolvable.
 - brew's `llvm` formula does not ship `ld.lld` at all. The separate `lld`
 formula does.
Measured on an arm64 Darwin host with llvm 22.1.8 and lld 22.1.8 both installed:
`/opt/homebrew/opt/llvm/bin/ld.lld` is ABSENT, `/opt/homebrew/opt/lld/bin/ld.lld`
is present and symlinked onto PATH. So the step installed the wrong formula and
then checked a path that formula would not have created either way.
Fixed: install `lld`, and assert via `command -v ld.lld` + `ld.lld --version`
rather than a reconstructed prefix. The assertion still REQUIRES the tool rather
than probing for it — the differential itself skips its ld.lld reverse-refusal
control when the tool is missing, and a skipped red-first control is exactly the
silent-skip class this repo keeps finding. We control this runner, so absence is
an error and the control stays non-vacuous.
Verified: the corrected assertion runs green on an arm64 Darwin host
(`uname -m` arm64, `uname -s` Darwin, `command -v ld.lld` resolves,
`ld.lld --version` prints LLD 21.0.0); ci.yml parses at 59 jobs.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
...place; sweep the class, not the instance
The previous commit fixed `"$(brew --prefix llvm)/bin/ld.lld"` in the job's
ASSERTION step and left the identical expression in the step that actually runs
the oracle:
 SYNTH=./target/debug/synth REQUIRE_NATIVE=1 LLD="$(brew --prefix llvm)/bin/ld.lld"
So the assertion went green and the oracle then failed one step later — `LLD`
pointed at a binary that does not exist, the ld.lld reverse-refusal probe did not
run, `host-linker refusals` came back 2 instead of 3, and the step's grep failed.
Fixing the instance instead of the class is the v0.56 lesson (#932/#953/#959,
where the same sentinel bug was patched three releases running before anyone
swept all 54 sites).
SWEPT, and the sweep found one more use that is CORRECT and is deliberately left
alone: `OBJCOPY="$(brew --prefix llvm)/bin/llvm-objcopy"` in
`aarch64-native-matrix`. Measured on an arm64 Darwin host with llvm 22.1.8
installed — the llvm formula ships `llvm-objcopy` and `clang` but NOT `ld.lld`,
which lives in the separate `lld` formula. So the pattern is not uniformly wrong
and a blanket replace would have been its own defect.
Fix: drop the LLD override entirely. `brew install lld` (previous commit) puts
`ld.lld` on PATH, which is where the differential's `find_tool` looks by default.
VERIFIED by reproducing the CI invocation locally on arm64 Darwin, with no LLD
override, exactly as CI will now run it:
 byte-identical modules: 141
 acceptance disagreements: 0
 identity mutation: detected
 executions: 17
 host-linker refusals: 3
 native-abi runs: 1
 mutation: detected
 RESULT: PASS
The run also answers a question #1180 left open: two synth Mach-O objects DO
collide on `_func_1` exactly as the ELF pair does, so the `func_N` STB_GLOBAL
shape is shared across containers rather than ELF-specific. The oracle pins that
as a limitation rather than asserting the opposite.
ci.yml parses at 59 jobs.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
... the ARM sweep's EXPECTED_DECLINES
Same shape and same cause as its arm64-Linux sibling in #1179. The ARM corpus
sweep compiles EVERY `.wat` under scripts/repro for ARM Thumb-2, and this PR adds
an aarch64 fixture carrying an `f64_scale` export. On the ARM default target that
hits GI-FPU-002 phase 2 (scalar f64 requires a double-precision FPU), so
`--all-exports` would ship a partial object and synth correctly REFUSES under
#952. Not a regression — the gate noticing a new fixture with no expectation.
Measured against the branch binary at `--target cortex-m4`, not inferred:
 warning: skipping function 'f64_scale': ... GI-FPU-002 phase 2: scalar f64
 requires a double-precision FPU
 Error: #952: 1 of 12 requested export(s) were skipped ... f64_scale
The sweep's RATCHET arm reddens if an EXPECTED_DECLINES entry ever starts
compiling, so the entry cannot rot silently — which is why this is the right
mechanism rather than excluding the file from the sweep.
Verified on the rebased tree: `#973 ARM CORPUS SWEEP: PASS` (compiled 158/177,
executed 2526/2526, mismatches 0), oracle_wiring exit 0 at the pinned floor
324847 with 190 scripts / 183 wired / 0 unwired debt, claim_check 60/60,
status_evidence exit 0, ci.yml parses at 59 jobs, rivet at main's 40-error
baseline.
Refs #242
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /