diff --git a/.agents/docs/2026-09-10-596-verify.sh b/.agents/docs/2026-09-10-596-verify.sh index 43b33c95a..30c510a70 100755 --- a/.agents/docs/2026-09-10-596-verify.sh +++ b/.agents/docs/2026-09-10-596-verify.sh @@ -33,7 +33,7 @@ set -u VER="${MCPP_VERIFY_VERSION:?set MCPP_VERIFY_VERSION}" STORE="${MCPP_VERIFY_BIN:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp}" SENTINEL_VER="${MCPP_VERIFY_SENTINEL:-0.0.2}" -COMPAT_VER="${MCPP_VERIFY_COMPAT:-2026年09月10日}" +COMPAT_VER="${MCPP_VERIFY_COMPAT:-2026年09月11日}" fails=0 skipped="" diff --git a/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md new file mode 100644 index 000000000..d4fccd0f6 --- /dev/null +++ b/.agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md @@ -0,0 +1,1197 @@ +--- +subject: plugins +status: active +--- + +# The category the plugin taxonomy does not name, and what a platform actually decomposes into + +Date: 2026年09月11日. Base: mcpp `2d102817`, `mcpp-plugins` `b4f7590` (0.2.6). + +**How to read this document.** Sections 1 to 3 are analysis of the base commits +above and can be checked against them. Sections 4 to 7 are a proposal and are +not implemented. Section 8 is what the proposal does not solve. + +The thesis is one sentence: **the engine should own the mechanism by which a +distributable is produced, and no format should live in the engine at all.** + +The occasion is a framework outside this project — HuxerUI, a declarative UI +library with a CMake build and a six-platform reach — being ported to mcpp, and +its ecosystem library `Lib-Live2D` being examined as the case that a package +manager either serves or does not. Two things came out of it that are about +mcpp rather than about that framework, and this document is those two. + +## 1. What is already true + +`mcpp:plugins` 0.2.6 is one package whose members are selected by features: + +```toml +[features] +default = [] +rules-ascendc = { sources = ["rules/ascendc.cppm"] } +rules-cuda = { sources = ["rules/cuda.cppm"] } +rules-hip = { sources = ["rules/hip.cppm"] } +rules-spirv = { sources = ["rules/spirv.cppm"] } +rules-sycl = { sources = ["rules/sycl.cppm"] } +tools-embed = { sources = ["tools/embed.cppm"] } +``` + +The taxonomy behind the two prefixes is stated in `docs/30-build-mcpp.md:789`: + +> **A tool is not a rule.** A rule states how a translation unit is compiled by +> a compiler mcpp does not drive: it submits an action and the engine schedules +> it. A tool states something the build program needs that no compiler +> performs, and does it while the program runs. + +And the reserved prefix, from the same file: + +> `warning: build rule 'mcpplibs.plugins' declares the module 'mcpp.rules.spirv'; +> the 'mcpp.' prefix is reserved for rules maintained by the mcpp project.` + +Five extension points do the extending, and `docs/31-authoring-a-rule-package.md` +says what they buy: **the engine holds no name that comes through any of them**. +Slang is the evidence — "the first language mcpp supports without naming it in +the engine". + +## 2. The category the taxonomy does not name + +A third kind of work fits neither definition. It does not compile a translation +unit, so it is not a rule; and it does not run while the build program runs, so +it is not a tool. It consumes **link outputs** and produces something a user +installs: + +| Work | Compiles a TU? | Runs in the build program? | +|---|---|---| +| SPIR-V from a shader | yes | no | +| A data file into a header | no | yes | +| **An MSI from the linked program** | **no** | **no** | +| **codesign on a `.app`** | **no** | **no** | +| **A `.deb`, an AppImage, an `.apk`** | **no** | **no** | + +The engine already has the mechanism: `role = "artifact"`, whose inputs are link +outputs, so ninja sequences it after the link. `docs/30-build-mcpp.md:464` names +the intended uses — "codesign, packaging, size budgets". What is missing is a +name for the packages that ship such actions, and therefore a place for them. + +**This is not hypothetical.** The HuxerUI port implements a Windows installer +this way and it is green on Windows CI: one `role = "artifact"` action running +`wix build`, with the program passed as `-d Executable=${mcpp.target_file:}` +and the definition naming it with ``. The +resulting MSI is 1572 KB and its `File` table carries one row, the 6.9 MB +application. Nothing in that action is specific to a UI framework except a +default icon path. + +An earlier revision of the same action bound a directory (`-bindpath +Application=bin`) and harvested it. When the path resolved to nothing on +Windows, WiX produced a **valid, empty, 52 KB installer with no diagnostic**. +That failure is worth recording because it is the argument for a shared package +rather than a per-project action: the mistake is not obvious, it is silent, and +every project that writes its own installer step gets to make it once. + +### 2.1 `mcpp pack` keeps the universal format and dispatches the rest + +`mcpp pack` is the right home, and the question is which half of it is the +engine's. `docs/10-pack-and-release.md:389` documents `[pack]` as declarative +configuration over a fixed set of modes (`static`, `bundle-project`, +`bundle-all`, `system`), and lists `.deb`, `.rpm` and AppImage under **Planned +Support**. + +Those three should not land there, and the split that keeps them out is: + +> **`mcpp pack` owns the mechanism and the one universal format. Every other +> format lives in a package, and `mcpp pack` dispatches to it.** + +The universal format is what it already produces: an archive that extracts and +runs. It is universal in the only sense that matters here — it needs no +knowledge of anyone else's release. Everything past it does. dpkg's control +fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's +signing scheme v3: each one bound into the engine couples an mcpp release to a +release mcpp does not control. The project already made this argument for +languages — Slang is supported "without naming it in the engine" — and a +distribution format has less claim to a name in the engine than a language +does. + +Dispatch is what keeps this from fragmenting into "packaging is a thing you do +outside `mcpp pack`" — and **the flag for it already exists**: + +``` +mcpp pack --format tar (default; .zip for a Windows target) | dir +``` + +`tar` and `dir` answer the same question `msi` and `appimage` answer — what +shape does the output take — so they belong on one axis, and the proposal is +not a new flag but a wider set of values for this one: + +```bash +mcpp pack --format msi +mcpp pack --format appimage +``` + +exactly as `--target` reaches a triple the engine did not have to know about +individually. Two consequences follow from the values no longer being a fixed +list: `--help` says "plus any format the resolved graph provides", and an +unknown value names what *is* available rather than a constant. + +### 2.2 What the engine must expose for that to work + +Three additions, and each is **format-neutral** — which is the test for whether +something belongs in the engine at all. + +**(a) A staged tree an artifact action can consume.** `mcpp pack` already +computes one: the dependency closure, the strip policy, the debug-symbol split, +`include`/`exclude`. It then compresses it and the directory is gone. A `.deb`, +an AppImage, a `.app` and an `.msi` all want exactly that directory and today +must each rebuild it. Exposing it — a `${mcpp.stage_dir}` placeholder, or a +`--stage-only` mode whose output path an action can name — is one addition that +serves every format, present and future, and encodes no format's knowledge. + +**(b) Package metadata in the build program.** A build program is told +`package_name()` and `package_namespace()` and not the version, description, +license or authors. Every installer needs the version; the HuxerUI +implementation therefore asks the project to restate it in the rule's options, +where it can drift from `[package] version` with nothing to detect it. This is +a read of data mcpp already parsed. + +**(c) `--format` resolves its value through the graph.** A package declares that +it provides a format name; `--format ` finds the provider among the +resolved dependencies and hands it the staged tree. The engine holds the +*dispatch*, not the format — the same shape as `device_extensions`, where the +engine classifies a source it cannot compile and a package supplies the +compiler. This is the smallest of the three additions, because the flag, its +parsing and its position in the command are already there. + +With (a) to (c), `[pack]`'s built-in modes become one provider among several +rather than the model, and nothing new needs to join them. + +## 3. Where a platform actually decomposes + +The second question was whether Android, iOS and Web can be reached from +packages. The answer differs per platform, and the difference is not a matter +of degree. + +A target is **not** extensible from a package. `modules/toolchain-model/src/triple.cppm` +declares the identity as three strings — + +```cpp +struct Triple { + std::string arch; // "x86_64" | "aarch64" | "riscv64" | ... + std::string os; // "linux" | "macos" | "windows" + std::string env; // "gnu" | "musl" | "msvc" | "" +}; +``` + +— with `os == "none"` added for freestanding, and a **23-row +`kKnownTargets` table compiled into the binary**: + +```cpp +{ "x86_64-windows-msvc", "verified", "PE", "", "", false }, +{ "aarch64-macos", "verified", "", "", "", false }, +{ "aarch64-linux-gnu", "planned", "", "", "", false }, +{ "riscv64-none-elf", "verified", "bare", "llvm@22.1.8", "xim:picolibc-riscv@1.8.12", true }, +``` + +Bare-metal targets are **rows in that table**, not something a board-support +package introduced; a BSP supplies the runtime for an already-known target. So +the engine boundary is exact: a package can add a language, a tool, an action, +a payload and a generated module. It cannot add a triple. + +There is no separate object-format model to extend either, and that is the +sharper form of the same fact. The binary format is not a field; it is derived +from `os` at each site that needs it — `is_pe()` is `os == "windows"`, +`is_freestanding()` is `os == "none"`, and `family()`, the `cfg()` dimension, +answers `windows` or `unix` and **nothing at all** for anything else. A fourth +answer to "what does this produce" is therefore not one +addition but an addition at every such site, which is why wasm is a different +size of change from a table row. (`nasm_format()` looks like an object-format +switch and is not one: it is NASM's `-f` flag, x86-only by construction, and it +hard-errors off x86 rather than choosing.) + +With that boundary fixed, each platform decomposes into layers, and only the +first is engine work: + +| Layer | Android | iOS | Web | Owner | +|---|---|---|---|---| +| **Toolchain does modules** | **yes, with a supplied surface + one define** (3.1) | not measured | **yes, with a supplied surface** (3.1) | **prerequisite, and answered for two** | +| Target identity | `env = "android"`, one row | `os = "ios"`, one row | **new arch, new os, new object format** | **engine** | +| Toolchain payload | `xim:android-ndk` | `xim:iphoneos-sdk` | `xim:emsdk` | index | +| Sysroot | table column, or `[target.
].sysroot` | same | same | engine / manifest | +| Compile and link flags | `-target aarch64-linux-android` | `-miphoneos-version-min` | `-sUSE_WEBGL2` etc. | **plugin** | +| Packaging | `.apk` | `.app` | `.html` + `.wasm` + `.js` | **plugin** | +| Signing | `apksigner` | `codesign` | — | **plugin** | +| Running | `adb install` | simulator | a browser | **plugin** (`mcpp::runner`) | +| Non-C++ glue | Gradle | Xcode project | JS bridge | **plugin** | + +The distances are unequal, and the ranking is the opposite of the usual demand +ranking: + +- **Android is the smallest.** `aarch64-linux-gnu` is already a `planned` row, + ELF is already an object format, and `aarch64` is already an arch. What is + missing is an `env` value and a sysroot that points at an NDK. +- **iOS is next.** `aarch64-macos` is `verified`, so Mach-O and the Apple half + of the toolchain model exist. What is missing is an `os` value and the + iPhoneOS SDK. +- **Web is the outlier.** A new arch (`wasm32`), a new os, a new object format, + and a driver that is a wrapper rather than a compiler mcpp drives directly. + This is [#597](https://github.com/mcpp-community/mcpp/issues/597), and it is + the only one of the three that changes the target model rather than extending + a table. + +**The consequence for planning:** every layer below the first is the same +mechanism on all three platforms. A `.apk` step, a `.app` step and an +`.html`+`.wasm` step are three members of the category section 2 names, and +none of them waits on the engine to be *written* — only to be *useful*. + +## 3.1 The prerequisite nobody lists: does the platform's toolchain do modules? + +Section 3 treats the target row as the engine's part of the work. That is true +and incomplete. mcpp is module-first — `import std` availability is one of the +eleven fingerprint inputs, and a package's interface is a BMI — so a target row +for a toolchain that cannot compile a module interface unit would resolve, build +nothing, and be worse than its absence. + +This was measured on this machine rather than assumed, and the result is better +than the shipped state of either toolchain suggests: **`import std` works on +both Android and Emscripten today, and neither needs a fork or a compiler +upgrade.** What both need is a directory their vendor chose not to install. + +### The gap, stated once — and closed by one vendor since + +An LLVM installation that supports `import std` carries a generated module +surface beside its headers. + +**The NDK ships one as of r30.** Measured 2026年09月11日 against +`android-ndk-r30-linux.zip` (`Pkg.Revision 30.0.16248370`, the current LTS): +`share/libc++/v1/` holds `std.cppm`, `std.compat.cppm`, 110 `std/*.inc` and 21 +`std.compat/*.inc` — the same 133 files this section measured as **absent** on +r27, generated by Google from the same AOSP checkout as the compiler. So the +table below is r27's state, and for Android the remedy it describes is no +longer needed: there is nothing to fetch from an `llvmorg-*` tag, nothing to +substitute, and no second asset to publish. + +Two things about r30 did **not** change, and both matter more than the file +count: + +- **`-D__BIONIC_CTYPE_INLINE=` is still required.** The bionic ctype defect + reproduces verbatim on r30 / clang 21: 28 errors, confined to `cctype.inc` + and `locale.inc`, `using declaration referring to 'isalnum' with internal + linkage cannot be exported`. Google ships the surface without shipping a + build of it that works out of the box. +- **The version to match is still not derivable from the compiler.** r30's + `_LIBCPP_VERSION` is `210000` from clang `21.0.0 (based on r574158c)` — an + AOSP `toolchains/llvm-project` mirror revision, **not** a build of any public + `llvmorg-21.1.x` tag. So the pairing check below cannot compare against an + upstream tag for Android; it can only pin the value and refuse a change. + +That is the second time a guess in this section was wrong in the same +direction: **a vendor had already done the work and nobody looked.** The first +was Apple (below). The table is kept as the r27 measurement it was, because the +argument it supports — that a target row for a toolchain without the surface +would be worse than its absence — is what motivated looking at all. + +| | `xim:llvm` 20.1.7 | NDK r27 | Emscripten 4.0.19 | +|---|---|---|---| +| clang | 20.1.7 | 18.0.1 | 22.0.0git | +| `_LIBCPP_VERSION` | 200100 | 180000 | **200100** | +| `_LIBCPP_ABI_NAMESPACE` | `__1` | `__ndk1` | `__2` | +| `share/libc++/v1/std.cppm` | yes | — | — | +| `std/*.inc` + `std.compat/*.inc` | 110 + 21 | 0 | 0 | +| total | 133 files, 620 KB | **0** | **0** | + +The NDK additionally sets `_LIBCPP_HAS_NO_STD_MODULES` in its `__config_site`. +That macro appears **exactly once in the whole NDK — in `__config_site` +itself**; no header consults it. It disabled the *installation* of the module +files at libc++ build time and gates nothing in the library, which is what makes +supplying them legitimate rather than a workaround around a disabled feature. +Emscripten does not set it at all. + +### Why "just upgrade libc++ to 22" is not the answer + +Tried, and it fails for a reason worth recording. libc++'s headers are **not +portable across configurations**: `__config_site` is generated per build and +records the ABI, threading and locale decisions that build made. Pointing NDK +clang at llvm 22.1.8's headers gives + +``` +__config:13:10: fatal error: '__config_site' file not found +``` + +and the three `_LIBCPP_ABI_NAMESPACE` values in the table above are the deeper +form of the same fact: `__1`, `__ndk1` and `__2` are deliberately incompatible, +so replacing a vendor's libc++ renames every `std` symbol's ABI namespace. +"Upgrade to 22" therefore means *building* libc++ from source for that target +and accepting a platform ABI change — not a file swap. It is a real option for +a project that already static-links its C++ runtime, and it is not needed, +because matching the revision works. + +### Android: measured, works, one define + +Named modules work as shipped: + +``` +$ clang++ --target=aarch64-linux-android24 -std=c++20 --precompile m.cppm # OK +$ clang++ --target=aarch64-linux-android24 -std=c++20 -fmodule-file=m=m.pcm -c main.cpp # OK +``` + +`import std;` does not — `fatal error: module 'std' not found` — for the reason +the table gives. Supplying the surface is mechanical, because `std.cppm` is +generated: + +``` +// WARNING, this entire header is generated by utils/generate_libcxx_cppm_in.py +``` + +from `libcxx/modules/std.cppm.in` and one CMake substitution that fills +`@LIBCXX_MODULE_STD_INCLUDE_SOURCES@` with `#include` lines naming the +`std/*.inc` partitions. Taking `libcxx/modules/` from **llvmorg-18.1.8** — the +release matching `_LIBCPP_VERSION 180000` — and performing that substitution +reproduces the vendor's own 133 files. + +Compiling it against the NDK's own headers then failed, and the failure is the +interesting part: + +``` +std/cctype.inc:11: error: using declaration referring to 'isalnum' with + internal linkage cannot be exported +bionic ctype.h:127: __BIONIC_CTYPE_INLINE int isalnum(int __ch) { ... } +``` + +28 errors, all of that one kind, confined to 2 of the 110 partitions +(`cctype.inc` and `locale.inc`). bionic defines the ctype functions +`static __inline`, and a using-declaration naming an internal-linkage entity +cannot be exported from a module. + +bionic anticipated this. The macro is overridable and says why: + +```c +/* All the functions in this file are trivial ... we inline them by default. + * This macro is meant for internal use only, so that we can also provide + * actual symbols for any caller that needs them. */ +#if !defined(__BIONIC_CTYPE_INLINE) +#define __BIONIC_CTYPE_INLINE static __inline +#endif +``` + +With `-D__BIONIC_CTYPE_INLINE=` on the `std.cppm` compile, the surface builds +(27.5 MB BMI) and a program that uses it links: + +```cpp +import std; +int main() { + std::vector v{3,1,2}; + std::ranges::sort(v); + return std::format("{}-{}-{}", v[0], v[1], v[2]) == "1-2-3" ? 0 : 1; +} +``` + +``` +app: ELF 64-bit LSB pie executable, ARM aarch64, interpreter /system/bin/... +``` + +### Android execution: measured, and the answer is conditional + +The binary above was not executed when this was first written. It has been +since, and the result is split in a way that decides the row's tier rather than +merely informing it. Measured 2026年09月11日 with NDK r30 and `qemu-aarch64` 8.2.2: + +| | static | dynamic (the row's default) | +|---|---|---| +| `x86_64-linux-android` | **runs**, direct execution, no wrapper | does not run off-device | +| `aarch64-linux-android` | **runs** under plain `qemu-aarch64`, no flags | does not run off-device | + +Both confirmed with the `import std` program, printing `1-2-3`. + +**Dynamic fails for two independent, compounding reasons**, and neither is a +missing flag: + +1. **The loader is device-only.** Every dynamic artifact carries + `interpreter /system/bin/linker64`. `-L` and `QEMU_LD_PREFIX` affect library + SEARCH, not the interpreter path, so qemu opens the literal absolute path and + reports `Could not open '/system/bin/linker64'`. An exhaustive search of the + 2.3 GB installed NDK finds **zero** files named `linker` or `linker64`: the + Android dynamic linker ships only inside a system image. +2. **Even with a loader, bionic's libc is inert.** `libc.so`'s `.text` is 9176 + bytes for 1147 exported functions — about eight bytes each — and every + function inspected disassembles to exactly `bti c; ret`. That is the + documented NDK stub shape: it exists so the linker can resolve names, and + the real bodies come from the device. For contrast `libc++_shared.so` is + real code, 724 KB, because it is the one runtime the NDK is responsible for. + +**So the honest claim is "runnable only when statically linked", and the tier +follows from the DEFAULT configuration rather than from the best case.** The +rows carry `defaultStatic = false`, and that is correct about the platform: a +real Android application links dynamically against the device's bionic, and +setting the flag true to make a CI check pass would misdescribe the platform to +flatter the measurement. Android is therefore `preview`, with execution +recorded for a configuration that is not the default — which is more than +`preview` usually means and less than `verified` requires. + +`x86_64-linux-android` needs no runner at all for the static case; mcpp's +"no runner declared, attempt direct execution" default is already right. +`aarch64-linux-android` needs `runner = ["qemu-aarch64"]` and nothing else. + +What dynamic execution would take is an emulator harness rather than an +addition: a system image per (ABI, API level) at multiple gigabytes each, +`adb`/`emulator`/`avdmanager` on top of the NDK, KVM or nested virtualisation +that an ordinary runner does not have, and boot times in tens of seconds +against qemu-user's near-instant start. That is its own package and its own CI +lane. + +### The state as of 2026年09月11日, all three vendors measured at current versions + +The sections above are the 2026-09 measurement of the versions then current +(NDK r27, Emscripten 4.0.19). Every one of them was re-taken while building the +payloads, and the picture changed: + +| | NDK **r30** | Emscripten **6.0.9** | iPhoneOS **26.5** | +|---|---|---|---| +| `_LIBCPP_VERSION` | 210000 | 220108 | 210106 | +| a public `llvmorg-*` tag? | **no** — AOSP `r574158c` | yes, 22.1.8 | yes, 21.1.6 | +| ships the module surface | **yes**, 133 files | **yes**, 134 files | **no**, 0 files | +| needs a define to build it | `-D__BIONIC_CTYPE_INLINE=` | none | none | +| execution reachable here | see below | **yes** — node | no | + +**Two of the three vendors now ship the surface themselves**, and the third's +is derivable because its version is public. So the generation machinery this +section describes at length is needed for one target rather than three, and +what the recipes actually owe is narrower than the section implied: **pin the +`_LIBCPP_VERSION` you measured and refuse a change.** For Android that is all +it can be — an AOSP mirror revision has no upstream tag to compare against. + +The other corrections worth carrying: + +- **Emscripten's version was wrong in this document, in the direction the + document warned about.** It recorded `_LIBCPP_VERSION 200100` and clang + 22.0.0git; 6.0.9 reports `220108` and clang 24.0.0git. The lesson it drew -- + that the surface must match the LIBRARY and not the compiler -- is right, and + the numbers it drew it from are two releases stale. Re-measured from an + installed payload on 2026年09月11日 by preprocessing `_LIBCPP_VERSION` out of + `<__config>` with the payload's own `em++`, so the number is the library's + and not a release note's. + +- **The file count was 133, not 134**, and the two are the same surface size + the NDK ships: 2 `.cppm` plus 131 `.inc` partitions under + `/emscripten/cache/sysroot/share/libc++/v1/`. Counted rather than + recalled, because this document uses these counts as the evidence that a + vendor ships the surface. + +- **"No additional flags" needed one qualification, and the qualification is + the good news.** `em++` ships the surface's SOURCE, not a BMI, so + `import std;` on its own fails with `module 'std' not found`. What is not + needed is the *generation* machinery this section describes at length -- + there is nothing to generate. Building the BMI is the engine's ordinary job + for every toolchain it supports, and measured with exactly the two steps it + already performs: + + em++ -std=c++23 --precompile /share/libc++/v1/std.cppm -o std.pcm + em++ -std=c++23 -fmodule-file=std=std.pcm -o hello.js hello.cpp std.pcm + node hello.js -> 1-2-3 + + `std.pcm` is 34 MB and the link produced `hello.js` (65370 bytes) plus + `hello.wasm` (447175 bytes). `-Wno-reserved-module-identifier` is NOT + required -- the precompile succeeds without it, with two warnings, and mcpp + passes it only to silence them (`src/toolchain/clang.cppm:243`). + + So `wasm32-emscripten` needs no new standard-library mechanism at all: + `stdModuleSource` points at that `std.cppm` and the existing path takes it + from there. **And the two-file output is confirmed** -- which is the one + genuinely new engine item, answered by the implicit-output channel the link + edge already has for import libraries and PDBs. +- **`emsdk` cannot be a payload at all.** `emscripten-core/emsdk` publishes + **zero** GitHub releases, and its `emsdk.py` fetches the real toolchain over + the network at install time. The recipe therefore names what `emsdk.py` + itself downloads: an `emscripten-releases-builds` bundle on Google Cloud + Storage, addressed by a 40-hex commit hash and thus immutable. The `latest` + alias moves several times a week and is resolved once, in the recipe, rather + than at install time. +- **`em++` needs a Python interpreter before it opens any config file**, being + a `/bin/sh` wrapper that execs Python. And `NODE_JS` is mandatory for + *linking*, not only for running the result: measured, a broken `NODE_JS` + survives `-c` and fails the link on `tools/compiler.mjs`. + +### Emscripten: measured, works, no define + +Named modules work as shipped. The module surface was absent in 4.0.19 as it +was on Android, and the version to match is **not** the one the compiler +reports: `em++` was clang 22.0.0git while its libc++ was `_LIBCPP_VERSION +200100`, LLVM 20.1. llvm 22.1.8's surface therefore failed on `'flat_set' file +not found`; llvm 20.1.7's built with **no additional flags at all** (31 MB +BMI). Both halves of that have since moved -- see the table above. + +End to end, and this one did run: + +```cpp +import std; +int main() { + std::vector v{3,1,2}; + std::ranges::sort(v); + std::print("{}-{}-{}\n", v[0], v[1], v[2]); +} +``` + +``` +$ em++ -std=c++23 -fmodule-file=std=std.pcm -c app.cpp && em++ app.o std.pcm -o app.js +$ node app.js +1-2-3 +``` + +`app.wasm` is 470 KB. So the Web question splits cleanly in two, and only one +half is open: **the standard library story is answered**, and what remains is +[#597](https://github.com/mcpp-community/mcpp/issues/597)'s target model. + +### Apple: measured after all, and every guess in this section was wrong + +This section first said iOS could not be measured on a Linux host, that Apple's +libc++ is not a build of a public revision, and that the remedy might therefore +not exist. **All three are false**, measured 2026年09月11日 against +`iPhoneOS26.5.sdk` (49 MB, from a public mirror) and `xim:llvm@22.1.8`: + +| | | +|---|---| +| `_LIBCPP_VERSION` | **210106** — a public revision, and `llvmorg-21.1.6` exists upstream | +| `_LIBCPP_ABI_NAMESPACE` | `__1` — upstream's own default, not a vendor-private namespace like the NDK's `__ndk1` | +| `_LIBCPP_HAS_NO_STD_MODULES` | `/* #undef */` — Apple does **not** disable std modules, unlike the NDK | +| `std.cppm` in the SDK | **0 files**, exactly as on Android and Emscripten | + +So the recipe is the same recipe: take `libcxx/modules/` from the tag matching +`_LIBCPP_VERSION`, perform the `@LIBCXX_MODULE_STD_INCLUDE_SOURCES@` +substitution, and get 133 files and 620 KB — the same count and size the other +two produce, and the same the vendor ships where it ships one at all. + +Carried end to end, from a Linux host: + +``` +$ clang++ --no-default-config --target=arm64-apple-ios18.0 -isysroot \ + -nostdinc++ -isystem /usr/include/c++/v1 -std=c++23 \ + --precompile surface/std.cppm -o std.pcm # 34 MB BMI +$ clang++ ... -fmodule-file=std=std.pcm -c app.cpp # import std; OK +$ clang++ ... -fuse-ld=lld app.o std.pcm -lc++ -o app +$ file app +app: Mach-O 64-bit arm64 executable, flags: +``` + +The binary was not executed — no device and no simulator on a Linux host — so +this is "compiles and links", which is what the `preview` tier means. + +**`--no-default-config` is load-bearing, and the reason generalises past iOS.** +`xim:llvm`'s payload ships a `bin/clang++.cfg` that injects the host's glibc and +libc++ unconditionally: + +``` +-isystem /include/c++/v1 +-isystem /include +-Wl,--dynamic-linker=/lib64/ld-linux-x86-64.so.2 +``` + +Those apply to **every** target the driver is pointed at, so a cross target that +is not Linux/glibc silently gets the host's C and C++ standard library ahead of +its own. `-nostdinc++` does not displace them — measured: with `-isysroot` and +`-nostdinc++` both given, the search list still began with the host's +`include/c++/v1` and the compile failed inside the host's `stdint.h` on +`gnu/stubs-32.h`. The criterion is `clang -v`'s search list, not whether the +compile succeeds, because a compile that reads the wrong standard library +usually succeeds. + +That is a property of the payload rather than of iOS, and it applies to any +non-Linux target driven through it. + +### What this means for xim-pkgindex + +The index has **253 recipes and none of the three**: no `android-ndk`, no +`emsdk`, no `iphoneos-sdk`. `llvm` is there and is the shape to copy. + +| Recipe | Beyond the archive it must carry | Blocked on | +|---|---|---| +| `xim:android-ndk` | `share/libc++/v1/` from llvmorg-18.1.8, a `_LIBCPP_VERSION` check, and `-D__BIONIC_CTYPE_INLINE=` on the std BMI | **nothing — measured working** | +| `xim:emsdk` | `share/libc++/v1/` from the release matching *libc++*'s version, not the compiler's | **nothing — measured working** | +| `xim:iphoneos-sdk` | licensing decides whether it installs or merely locates | a licence reading, and one measurement | + +Both writable recipes owe the same two things: **pin both halves to one +revision and refuse on mismatch** — the check is exact, `_LIBCPP_VERSION` in +`__config_site` against the release the surface came from — and **re-derive on +every vendor bump**, since the surface is a function of the vendor's libc++, +not a constant. + +The licensing asymmetry is worth stating plainly. The NDK is Apache-2.0 and +Emscripten is MIT, both redistributable; the iPhoneOS SDK is neither, which is +why cross-platform toolchains reach it through a locally installed Xcode. +`xim:iphoneos-sdk` may therefore have to be a *locator* — a recipe that finds +and pins what the machine already has, the way `msvc@system` does — rather than +an installer. mcpp already has that shape. + +## 4. The members this proposes + +Nothing here is a new repository. `mcpp-plugins` exists and its shape — one +package, one module interface unit per feature — already fits. + +| Feature | Module | Category | What it does | +|---|---|---|---| +| `dist-wix` | `mcpp.dist.wix` | dist | An MSI from a linked program and a definition it renders. Windows only. | +| `dist-appimage` | `mcpp.dist.appimage` | dist | An AppImage from a staged tree. Linux only. | +| `dist-apple` | `mcpp.dist.apple` | dist | A `.app` bundle, `Info.plist`, and `codesign`. macOS now; iOS when the row exists. | +| `dist-android` | `mcpp.dist.android` | dist | An `.apk`: Gradle invocation or direct `aapt2`/`d8`/`apksigner`. When the row exists. | +| `dist-web` | `mcpp.dist.web` | dist | The `.html`/`.js`/`.wasm` set and its loader. When the target model admits wasm. | + +Two of these can be written **today**, against the engine as it is: `dist-wix` +(a working implementation exists and would be a port, not a design) and +`dist-appimage`. `dist-apple` can be written for macOS today and gains iOS when +the row lands. The other two wait on section 3's first layer. + +### Why `dist-` and not `rules-` + +Because the taxonomy in section 1 is load-bearing and these are not rules. A +consumer reading `rules-wix` would expect a compiler it does not drive and a +translation unit; there is neither. The prefix should say which of the three +questions the member answers: + +``` +rules-* how is this translation unit compiled +tools-* what does the build program need to do itself +dist-* what comes out of the link, and in what form a user installs it +``` + +`mcpp.` stays reserved for members of this repository, unchanged, and +`mcpp.build.*` remains the engine's own module family — `mcpp.dist.*` collides +with neither. + +## 5. What `tools-embed` is missing + +`mcpp.tools.embed` covers one file at a time. `files()` writes **one header per +input**, deriving an identifier from each, and refuses `options::identifier` +because it "names one symbol and `files()` writes several". + +The case it does not cover, taken from Lib-Live2D's `cmake/EmbeddedShaders.cmake` +(34 lines of `file(READ)` and string concatenation, with a second 59-line +variant for Metal): **N inputs, one header, one table**, where each row carries +the file's name alongside its contents and the consumer iterates. + +```cpp +inline constexpr EmbeddedShader embedded_shaders[]{ + {"Standard.vert", R"(...)"}, + {"Standard.frag", R"(...)"}, +}; +``` + +This is not a new plugin. It is a `table()` entry point beside `file()` and +`files()`, with an option for the row type's name and for how the key is +derived from the path. The `write_if_different` behaviour already there is the +part that makes it cheap to call unconditionally, and it carries over. + +I have not measured how many projects want the table shape rather than the +per-file shape. One does, and its 93 lines of CMake are the evidence that the +shape is worth having; that is an argument for adding it, not a measurement of +demand. + +## 6. What a dist member owes its consumer + +The rules in `docs/30-build-mcpp.md` apply unchanged, and two of them bind +harder here than for a rule: + +**Expose a plan/submit pair.** A dist member's output is the last thing before +a user's hands, so it is the most likely to need a project-specific edit — a +different compression level, an extra file, a second signature. `generate_all(opt)` +being `submit(plan_all(opt))` is what keeps that edit from becoming a +reimplementation. + +**Failure and advice use different channels.** A packaging step that succeeds +while carrying nothing is the failure mode section 2 measured. Where a dist +member can detect an empty or implausible result, it must say so on a +*successful* build through `mcpp::warning`, because stderr on success is +discarded. + +**One `(name, version)` names one payload.** A dist member that wraps a signing +tool inherits that tool's compatibility surface. Versioning in lock-step with +the wrapped tool is legitimate and says something true. + +Two more that are specific to this category: + +**Name the input, do not harvest a directory.** Section 2's 52 KB installer is +the general case: a path that resolves to nothing is silent, and a named input +that is missing is an error. `${mcpp.target_file:}` is the mechanism — +a build program is told neither the triple nor the fingerprint, and an unknown +target name is refused rather than expanded to an empty path. + +**Declare the tool where it will be looked up.** `xpkg_dir` answers from +`MCPP_XPKG_*_DIR`, which mcpp sets for the *building* package. A dependency's +declaration provisions the payload — the log says +`Provisioning [xlings.workspace] entries (...)` — without making it visible to +a consumer's build program. A dist member that runs a payload tool therefore +declares it itself, and says so when the lookup returns empty rather than +pointing at a manifest the reader does not own. + +## 7. Staging + +The order is not the demand order, and the reason is that the lower layers are +shared: + +1. **`dist-wix`, then `dist-appimage`.** No engine change. `dist-wix` is a port + of a working implementation; `dist-appimage` is the same shape on the + platform where it is easiest to test. Both restage by hand, which is the + evidence for step 2 rather than a reason to delay them. +2. **The two engine additions of 2.1: a consumable staged tree, and package + metadata in the build program.** Both are format-neutral, and step 1 will + have shown what each costs to do without. After this, no format needs the + engine again. + +3. **`tools-embed`'s `table()`.** Independent of everything else, and the + smallest of these. +4. **`dist-apple` for macOS.** Establishes the bundle-and-sign shape on a target + that already exists, so that iOS later adds a row and not a design. +5. **`xim:android-ndk` and `xim:emsdk`, each carrying the module surface (3.1).** + Both are measured working and neither waits on the engine. Writable before + the rows, and doing them first turns each row into a small verifiable change + rather than a change plus an unknown. +6. **The Android row.** One `env` value, one table row, one payload. The + smallest of the three platform steps, and it makes `dist-android` writable. +7. **The iOS row**, then `dist-apple` extends to it — preceded by the same + measurement for Apple clang, and by whether `xim:iphoneos-sdk` can be a + payload or must be a locator. +8. **Web**, as [#597](https://github.com/mcpp-community/mcpp/issues/597) + describes. It is a target-model change and not a table row — but it is now + *only* that: 3.1 measured the standard-library half and it works, so #597 is + one problem rather than two. + +Steps 1 to 4 do not wait on a platform. Doing them first means that when a +platform row lands, the layer above it already exists — and that the row is the +only thing that had to land. + +## 8. What this does not solve + +**A published package still carries no consumer dependencies from the target +axis.** `mcpp emit xpkg` derives `xpm..deps` from the top-level +`[xlings.workspace]` only. A dist member declaring its tool on the target axis — +which is where a payload the produced code links against belongs — publishes +cleanly and fails in the consumer's link. The workaround is to declare on both +axes, and the second copy states a target fact on the host axis, which mcpp's +own guidance calls the wrong form. + +**`mcpp test` is not configurable.** A library with an existing test tree that +is not `tests/**/*.cpp` cannot use `mcpp test` at all; HuxerUI reached this and +answered it by adding a separate package whose `tests/` selects instead of +discovering. This is unrelated to plugins and is noted because it is the second +thing an ecosystem library hits. + +**Every guess along the way was wrong, including the ones this document made +about Apple.** That the gap was one file (it is 133); that a mismatched surface +fails obscurely (it names the missing header); that the blocker was libc++ (it +was bionic's `static inline` ctype); that Emscripten's libc++ version follows +its clang version (it does not); that iOS could not be measured from Linux (it +can); that Apple's libc++ is not a public revision (it is, 210106); and that +Xcode might already ship the surface (it does not, and the SDK carries none). + +The pattern is worth naming rather than just recording: **each wrong guess was +about what a vendor had done, and each was cheap to check and was not checked.** +The section that guessed least — Emscripten, where the version was read rather +than inferred — is the only one that needed no correction. + +**Neither working recipe was carried to a released payload.** Both were built +and exercised in a scratch directory. Turning each into an xim recipe — the +fetch, the substitution, the version check, the layout — is the next step and +is not done here. + +**Nothing here shortens the platform work itself.** Sections 3 and 7 say which +layer is engine and which is package; they do not make the engine layer +smaller. Android is small because the model nearly admits it already, not +because a plugin can stand in for the row. + +## 9. Review of sections 4 to 7, and what each risk is measured against + +The proposal survives review with one correction, one omission that would have +been a silent defect, and one boundary the sections above draw in the wrong +place. They are stated here before the task list because each one moves a task. + +### 9.1 The correction: `--stage-only` already shipped, under another name + +Section 2.2(a) asks for "a `${mcpp.stage_dir}` placeholder, or a `--stage-only` +mode whose output path an action can name". The second half exists. +`--format dir` sets `writeArchive = false`, and `mcpp pack` then reports +`plan.stagingRoot` — `target/dist//` — as the output. That tree +is the closure after the strip policy, the debug split and `include`/`exclude` +have run: the thing section 2.2(a) describes. + +So only the placeholder is new. `--stage-only` would be a second spelling of a +mode that ships, which is the shape this project refuses elsewhere +(`mcpp sbom` versus `mcpp emit sbom`). + +### 9.2 The omission: the requested format is a graph input, so it is graph state + +`mcpp pack --format appimage` changes what the build program submits. Anything +that changes what a build program submits changes the graph, and +`target/
//build.ninja` is shared mutable state that two fast paths +replay. `mcpp.build.graph_shape` exists because of exactly this class of +defect, and it already carries two such axes: `graph=` (a test graph replayed +for a plain build) and `accel=` (a device variant a flag chose, replayed for a +build that chose nothing). + +A `dist` axis with no entry on that line reproduces the same failure a third +time: `mcpp pack --format appimage`, then `mcpp build`, replays a graph +carrying a dist edge that a plain build must not have. The line therefore gains +a third field, and `is_plain_build_graph` requires it to read `none`. + +The format deliberately does **not** enter the build fingerprint. It would put +the packaging pass in its own directory and cost a full recompile to produce a +distributable from an already-built tree. The header line is the cheaper half +of that pair and is the half that answers the question the fast paths ask. + +### 9.3 The boundary in the wrong place: a staged tree is not a root filesystem + +Section 2.1 lists `.deb`, `.rpm` and AppImage as one group and 2.2(a) offers +one staged tree to all of them. Two shapes are being conflated: + +| Format | Wants | +|---|---| +| AppImage, `.app`, `.msi` | a **bundle** tree: `bin/`, `lib/`, relocatable, rooted anywhere | +| `.deb`, `.rpm` | an **FHS** tree: `usr/bin/`, `usr/lib//`, rooted at `/` | + +`mcpp pack` stages the first — it is what `--mode vendored` means, and the +`$ORIGIN` rewriting and the `run.sh` wrapper are what make it relocatable. A +`.deb` member consuming that tree must re-lay it out, and the re-layout is +`.deb`'s knowledge rather than the engine's, so this is not an argument for a +second staged tree in the engine. It is an argument about which member goes +first: **a bundle-shaped format exercises `${mcpp.stage_dir}` as it is, and an +FHS-shaped one exercises a re-layout step that would then be the thing under +test.** Section 7's choice of AppImage for step 1 is right, and the reason is +this rather than "the platform where it is easiest to test". + +### 9.4 The ordering problem sections 2.2 and 7 do not state + +A `role = "artifact"` action is a ninja edge. The staged tree is produced by +`mcpp::pack::run` in C++ **after** ninja has finished. So an artifact action +cannot depend on the staged tree in the pass that builds it, and +`${mcpp.stage_dir}` is not expressible in a single-pass pack. + +Two passes are, and every value the second one needs is already answered by the +first: + +1. `prepare_build`, with no format set. Build programs run and declare the + formats they provide; none submits a dist action, because none was asked + for. An unknown `--format` is refused **here**, before anything is compiled, + naming the set that was declared. +2. The ordinary build. The link outputs exist. +3. `make_plan` and `pack::run`. The staged tree exists at `plan.stagingRoot`. +4. `prepare_build` again, with `pack_format` and `pack_stage_dir` set to what + steps 1 and 3 answered — **not re-derived**. The claiming member submits its + action. Its command is what ninja runs. + +The re-derivation is what would have been the defect. `plan.stagingRoot` is a +function of the package name, the version, the resolved triple and the mode, +and the resolved triple is not known until prepare has run. Computing it a +second time before prepare — from `host_triple()`, say — is the shape where two +derivations of one value agree on every machine the author has and disagree on +one they do not. + +The second prepare is not free. It is bounded by the build program re-running: +the contract values are part of its re-run key unconditionally, so changing +`MCPP_PACK_FORMAT` invalidates exactly that one entry and nothing else. + +### 9.5 The declaration must not be gated on the request + +A member that emitted `mcpp:pack-format=appimage` only when +`pack_format() == "appimage"` would make the set unknowable: the engine could +never answer "which formats does this graph provide" and `--format bogus` could +name nothing. So the contract has two halves that must not be merged: + +> **Declare unconditionally. Submit conditionally.** + +This is the load-bearing rule of the whole dispatch, and it is the rule most +likely to be got wrong by a member author, because a member that gets it wrong +still works for the person who wrote it — they always pass their own format. +It therefore needs a test whose failure mode is the wrong half: a build that +requests **no** format and asserts the set is still non-empty. + +### 9.6 Risks, each with the measurement that would catch it + +| Risk | Why it would pass unnoticed | Criterion | +|---|---|---| +| The dist graph is replayed for a plain build | Both graphs live in one directory; the fast path predates any plan | `pack --format X`, then `build`, then assert no dist edge ran — the `A then B then A` shape | +| The second prepare replays a cached build program | A cached run re-emits the first pass's output, which submits nothing, so the pass succeeds and produces nothing | Assert the dist output **exists**, never that the command exited 0 | +| `${mcpp.stage_dir}` in a non-packing build | Expands to an empty string; the command then reads the build directory root, which exists | Refuse at expansion, and assert the refusal text | +| A member declares a built-in name (`tar`, `dir`) | The built-in wins and the member is silently unreachable | Refuse the collision, naming both | +| The dist action runs before staging | Only in a single-pass design; recorded so the two-pass ordering is not "simplified" away later | Assert the staged tree is non-empty **from inside the action** | +| A dist member's tool is absent | The tool is a host lookup, and an empty path becomes an argv token | Refuse in the build program, naming the tool and where it was looked for | +| The produced distributable is valid and empty | Section 2's measured 52 KB installer | Each member asserts a floor on its own output, on the **success** path, through `mcpp::warning` | + +### 9.7 The engine's half of section 3, and what is left after it + +Section 3's boundary is exact: a package can add a language, a tool, an action, +a payload and a generated module, and it **cannot add a triple**. Every layer +below the first — the `.apk` step, the `.app` step, the `.html`+`.wasm` step, +the runner, the signing, the non-C++ glue — therefore waits on a row in +`kKnownTargets` and on nothing else in the engine. So the rows are engine work +and belong in the same release as §2.2: + +| Row | Tier | What it still needs | +|---|---|---| +| `aarch64-linux-android`, `x86_64-linux-android` | `planned` | `xim:android-ndk` | +| `aarch64-ios` | `planned` | the iPhoneOS SDK, and a licence reading first (§9.8) | +| `wasm32-emscripten` | `planned` | `xim:emsdk` | + +**`planned` is a refusal, not a gap.** The tier gate answers `tier-planned` +naming the row, so `mcpp build --target aarch64-linux-android` says the +vocabulary has this target and nothing is wired yet — rather than `unknown +target`, which was false, or a build that resolves and produces nothing, which +§3.1 argues would be worse than the row's absence. Each cell is declared in +`tests/matrix/expected.tsv`, so the day a row is wired the matrix goes red and +says so. + +**Web needed one thing the other two did not, and it was not a table row.** The +binary format was never a field: it was re-derived from `os` at each site that +needed it, which is affordable while the answer has two values. `wasm32` is the +first target whose format is neither, and a third value turns those derivations +into an addition at every such site — where a missed site does not fail, it +silently answers ELF. `ObjectFormat` is that addition made once. This is the +substance of [#597](https://github.com/mcpp-community/mcpp/issues/597)'s +"changes the target model rather than extending a table", and with §3.1 having +answered the standard-library half, #597 is now one problem rather than two. + +**Android's placement is the modelling decision.** `env = "android"` on a +`linux` OS, not `os = "android"`: the kernel is Linux, so ELF, the `unix` +family and `nasm -f elf64` are already right, and an OS value would have made +every one of them wrong by default and required a new answer at each site. What +differs from `gnu` is bionic, the loader path and the SDK, which is what an +`env` value is for. + +What remains outside this pass, each blocked on something no engine work +supplies: + +- **`xim:android-ndk`, `xim:emsdk`** (§7 step 5). Both recipes are measured + working in a scratch directory (§3.1); turning either into a payload means + fetching and republishing a multi-gigabyte vendor toolchain with a derived + 133-file module surface. That is its own release, not a side effect of this + one — and the rows landing first is exactly §7's ordering argument, which + said doing the payloads first would make each row small. The rows turned out + to be the cheap half either way. +- **The iPhoneOS SDK.** Not measurable on a Linux host, and §9.8 gives the + decision procedure rather than the measurement. +- **`dist-android`, `dist-web`**. Each waits on its payload, not on its row. + +### 9.8 The iOS SDK: a three-tier policy rather than an open question + +Section 3.1 leaves `xim:iphoneos-sdk` as "licensing decides whether it installs +or merely locates", and §8 repeats it. The decision procedure is not a +measurement — it is a preference order, and stating it removes the open +question without taking the measurement: + +1. **Redistribute, if the licence permits it.** A published payload with a + GitCode mirror, like every other `xim` toolchain package. Publicly mirrored + SDK trees exist — `https://github.com/xybp888/iOS-SDKs` is one — and whether + this tier is reachable is a licence reading of the SDK itself, not of the + mirror. +2. **Fetch from upstream, without a CN mirror.** A recipe that downloads at + install time from the upstream URL and mirrors nothing. This is what a + licence that permits use but not redistribution allows, and declining the + mirror is the point: a mirrored copy *is* redistribution, so the tier is + defined by what it refuses to do. +3. **Locate what the machine already has.** The `msvc@system` shape: find, pin + and report an installed Xcode, install nothing. Correct under any licence, + and the only tier that cannot serve a machine without Xcode. + +Tier 3 always works and is therefore the floor, not the goal. The recipe should +reach for the lowest tier the licence allows and say in its description which +tier it took, because a consumer reading "locator" needs to know that is a +licence conclusion rather than an unfinished recipe. + +## 10. The task list + +Four repositories. One pull request each, in this order, because each depends +on the one above it being released rather than merely merged. + +### 10.1 `mcpp` — the three format-neutral additions + +| # | Task | Depends on | +|---|---|---| +| E1 | `MCPP_PKG_VERSION` / `_DESCRIPTION` / `_LICENSE` / `_AUTHORS` / `_REPO` in the build-program environment, with accessors | — | +| E2 | `MCPP_PACK_FORMAT` in that environment, and `mcpp::pack_format()` | — | +| E3 | The `mcpp:pack-format=` outlet, collected onto the plan | — | +| E4 | `${mcpp.stage_dir}`: expansion for Artifact actions, refusal elsewhere, the stage manifest as an implicit input | E2 | +| E5 | `--format` accepts a provided name; the refusal names the available set | E3 | +| E6 | The two-pass pack pipeline (§9.4) | E4, E5 | +| E7 | `dist=` on the graph header line; `is_plain_build_graph` requires `none` | E6 | +| E8 | `docs/10`, `docs/30`, `docs/31`, and a new `docs/35`; the `zh` mirror of each | E1–E7 | +| E9 | Unit tests for E3–E5, e2e for E6–E7, each with the §9.6 criterion | E1–E7 | + +### 10.2 `mcpp-plugins` — one member per verified platform + +| # | Task | Depends on | +|---|---|---| +| P1 | `tools-embed`: a `table()` entry point (§5) | — | +| P2 | `dist-appimage` → `mcpp.dist.appimage` | mcpp released, X1 | +| P3 | `dist-wix` → `mcpp.dist.wix` | mcpp released | +| P4 | `dist-apple` → `mcpp.dist.apple`, macOS half only | mcpp released | +| P5 | `MCPP_VERSION` in CI raised to the engine that carries E1–E7 | E1–E7 released | +| P6 | Plan-level tests on all three runners; end-to-end each on its own | P1–P4 | + +### 10.3 `xim-pkgindex` — the one payload this needs + +| # | Task | Depends on | +|---|---|---| +| X1 | `xim:appimagetool` | — | + +### 10.4 `mcpp-index` — publication + +| # | Task | Depends on | +|---|---|---| +| I1 | `mcpp:plugins@0.3.0` | P1–P6 released | + +The engine tasks are the only ones on the critical path. P1 and X1 do not wait +on anything. + +## 11. The plan read from nine angles + +Section 9 reviewed the proposal on its own terms. This section reads the +*implemented* result from the angles a reviewer would apply independently of it, +because each angle catches a different class of mistake and several of them +caught one. + +### 11.1 Architecture + +The load-bearing claim is that **the engine holds the dispatch and no format**. +It survives one test the proposal did not anticipate: a format that consumes +nothing. `dist-wix` packages one named program and never reads the staged tree, +which is what §6's own guidance recommends — and the first implementation of the +dispatch refused exactly that member, because it identified the distributable by +"which action named `${mcpp.stage_dir}`". The criterion was a property of the +*mechanism* rather than of the *request*. It is now "which artifact actions the +request introduced", which needs nothing of the member. + +The same mistake occurred one layer down and was found by a real macOS runner: +staging ran before the dispatch and its failure was fatal, so every dispatched +format was unreachable on a target whose built-in bundling is refused. Staging +is a service to the provider, not a precondition. + +Both are the same error in different clothes: **the engine deciding something on +the provider's behalf.** That is the failure this architecture is most exposed +to, because the whole point of it is that the provider decides. + +### 11.2 Stability + +Three axes now ride `build.ninja`'s header line — shape, schedule, device +variant — and `dist=` is the fourth. Each was added after the same defect: a +graph written for one purpose replayed for another, in a directory the two +share. The format deliberately does not enter the fingerprint, because it would +cost a full recompile to package an already-built tree; the header line is the +cheaper half of that pair and is the half the fast paths ask. + +The measurement that matters is the one that says the criterion is worth +having: on 2026年09月11日 a plain build after a pack pass regenerates the graph +*even with the field ignored*, so an end-to-end assertion would pass whether or +not the field works. The unit test is where the invariant is held. + +### 11.3 Elegance + +Two additions were withdrawn as duplicates of something that ships. +`--stage-only` is `--format dir`. And `rule_module` on a `dist-*` feature was +refused by the engine, correctly: that key means "the module that reaches a +rule" and implies `device_extensions`, which a member compiling nothing cannot +have. The `dist-*` members take the `tools-*` shape instead, and the consumer +writes one line more than a rule needs — which says something true. + +### 11.4 User experience + +`--format` gained values rather than a second flag, because `tar`, `dir`, `msi` +and `appimage` answer one question. An unknown value names what *is* available +rather than a fixed list, and the refusal arrives before anything is compiled. + +The failure mode this category is most exposed to is a step that succeeds while +carrying nothing — §2's measured 52 KB installer. Every member therefore +asserts a floor on the success path through `mcpp::warning`, because stderr on +a successful build is discarded. The first such floor was a size bound and was +wrong on its first real fixture: a stripped hello-world stages at 14999 bytes, +under a 16 KB bound, so a correct AppImage was reported as empty. A size is a +proxy for a question that can be asked directly. + +### 11.5 Compatibility + +The engine's rule is unchanged: no per-package floor exists, and the index-level +`min_mcpp` does not move for a package, because raising it makes the whole index +unreadable to clients stopped below it. What an older client gets is legible at +the point of use, and for these members it is the best case of that rule — +`mcpp::provides_pack_format` does not exist in an older engine's bundled module, +so a consumer fails at the `build.mcpp` **compile**, naming the missing +function, rather than at a link or in an artifact. + +### 11.6 Cross-platform + +Three members, three platforms, and the honest asymmetry is that only one of +them could be measured where it was written. `plan_for()` returning +`applies == false` on the wrong OS says the gate works and says nothing about +whether the tool accepts what the member renders. That gap was closed by adding +CI steps that actually run `wix build` and assemble a real `.app` — and the +first thing they did was fail, twice, for unrelated reasons: WiX 7 refuses to +run without an out-of-band licence acceptance (`WIX7015`), and the Mach-O +staging refusal above. Both are findings the plan-level assertions could not +have produced. + +### 11.7 Consistency + +`rules-*`, `tools-*` and `dist-*` is one taxonomy with one rule: the prefix +says which of three questions a member answers. The engine's own three-value +`ObjectFormat` is the same discipline applied to a fact rather than to a +package — the binary format was re-derived at roughly 35 sites, which is +affordable at two values and becomes an addition at every site at three, where +a missed site silently answers ELF. + +Two sites answered the object format by searching for `"apple"` in a string +that never contains it, so an explicit `--target aarch64-macos` — a *verified* +row — linked and recorded as ELF while a native build on the same machine said +`macho`. One function, two paths, only the exercised one right. + +### 11.8 Seamless upgrade + +Every new field is absent-tolerant in the direction that matters. An older +graph's missing `dist=` reads as a miss and never as `none`. A build program on +an older engine gets empty strings from the new accessors, which a member reads +as "fall back to what you did before". The `pack-format` directive carries a +non-empty cache tag, so a declaration survives a cache hit — the pass that +reads it is `mcpp pack`, which is never a project's first build, and an +unpersisted declaration would be absent exactly when a user names a format. + +`kCacheEpoch` is deliberately not bumped: an entry written before the row +carries no such line and the program that wrote it could not emit one, so +replaying it yields what that program said. + +Measured across two real binaries rather than only in a unit test, because an +absent-tolerance claim is about what a *previous version* wrote: + +| step | binary | the graph's header line | result | +|---|---|---|---| +| 1 | released 2026年9月10日.2 | `graph=normal;schedule=none;accel=default` | builds | +| 2 | 2026年9月11日.2 | `;dist=none` appended | fingerprint change, full rebuild, no error | +| 3 | 2026年9月11日.2 | unchanged | `Finished dev in 0.00s` -- the fast path replays | +| 4 | 2026年9月10日.2 again | its own older directory | `0.00s` -- the downgrade does not choke | + +Step 4 also says what the absent-tolerance is worth. The version is part of the +fingerprint, so two binaries never share a graph directory and an older mcpp +never actually reads a `dist=` field. That makes the field's read side defence +in depth rather than a live path -- which is the same conclusion §11.2 reaches +from the other direction, and is why the invariant is held in a unit test. + +### 11.9 Test coverage + +The count is not the measure; what each test excludes is. Two are worth naming. + +`638` holds nine properties, each paired with the wrong answer it excludes, and +two of them were verified load-bearing by removing the guard and watching the +test fail — including the one that distinguishes "the request introduced this +action" from "any artifact action". + +And one CI assertion was itself the defect: it grepped for `struct +embedded_file`, the *default* `row_type`, while the fixture sets +`row_type = "shader_entry"` precisely because that option exists. The code was +correct and the assertion was wrong, which is what a check tied to a spelling +the fixture chooses will eventually always be. It now reads the struct's name +out of the file and asserts the table's element type *is* that struct, for +every generated header rather than whichever `find` listed first. + +### 11.10 The dependency order, and why it is not the demand order + + mcpp engine ──────────────► released, because plugins CI pins a release + │ + ├─► xim payloads ─────► independent of the engine; merged first + │ + └─► mcpp-plugins ─────► needs the release, so it cannot precede it + │ + └─► mcpp-index ► needs the plugins tag's sha256 + +Four repositories, one pull request each. The engine is the only thing on the +critical path, and every measurement that changed the plan came from the layer +*above* it — which is the argument for doing the payloads early even though the +rows land last. diff --git a/.agents/docs/2026-09-11-platform-targets-design-review.md b/.agents/docs/2026-09-11-platform-targets-design-review.md new file mode 100644 index 000000000..f01581a58 --- /dev/null +++ b/.agents/docs/2026-09-11-platform-targets-design-review.md @@ -0,0 +1,1076 @@ +--- +subject: targets +status: active +--- + +# Where a platform's knowledge belongs: iOS, Android and Web across the engine, the index and the plugins + +Date: 2026年09月11日. Written after `wasm32-emscripten` reached `verified`, because +that row is the first of the three to be wired end to end and what it needed is +the evidence this review is about. + +The question: for iOS, Android and Web, is mcpp's division of target +identity, toolchain resolution and artifact shape the right one, and are the +names right. Answered against what four other build systems do, and against +the seven engine changes the wasm row actually required. + +## 1. What mcpp does today + +Three separable things, currently in three places. + +| | where | what decides it | +|---|---|---| +| target identity | engine, `kKnownTargets` | a closed table; a package cannot add a row | +| toolchain resolution | engine, `to_xim_package` + the gates | a property of the target | +| the payload itself | index, `xim:` | a licence question before a packaging one | +| artifact shape | plugins, `dist-*` | the format's own tool | + +The engine's stated rule is one sentence: **the engine owns the mechanism by +which a distributable is produced, and no format lives in the engine.** That +rule is about the fourth row. This document is mostly about the first two, +which the wasm work exercised for the first time. + +## 2. The comparison, and it is more reassuring than not + +### 2.1 Rust is the closest analogue, and the tiers line up almost exactly + +`rustc`'s target list is **closed** — `rustc --print target-list` — and carried +in the compiler, which is what `kKnownTargets` is. Its tier definitions: + +| Rust | guarantee | mcpp | +|---|---|---| +| Tier 1 | "guaranteed to work": official builds **and automated tests** | `verified` -- "built AND RUN" | +| Tier 2 | "guaranteed to build"; automated tests **are not always run** | `preview` | +| Tier 3 | code exists, no automated building or testing | `planned` | + +The mapping is close enough to be worth adopting as calibration rather than +coincidence. And it produces an uncomfortable measurement: + +**All three of these platforms are Tier 2 in Rust** — +`wasm32-unknown-emscripten`, `aarch64-linux-android` and `aarch64-apple-ios` +are each "guaranteed to build", with tests not always run. + +So mcpp's `verified` for `wasm32-emscripten` is a **stronger** claim than Rust +makes for its own wasm target, and mcpp's `planned` for Android and iOS is +**weaker** than Rust's. The honest end state for Android and iOS is +`preview` — buildable, not run in CI — and that is not a compromise, it is the +same claim the most comparable toolchain in the industry makes. + +`cargo` does **not** manage SDKs. `cargo-ndk`, `cargo-apk` and `xcodebuild` +are outside it. So Rust puts identity in the core and the SDK outside it, +which is mcpp's split with `xim:` in the "outside" position. + +### 2.2 Zig is the strongest counter-model, and it draws the same line + +Zig ships libc and headers for a closed list of 97 targets, and calls +cross-compilation "a first-class use case". It is the system most willing to +bundle a target's system — and it ships **neither the Apple SDK nor the +Android NDK**. Those remain the user's to provide. + +Two conclusions. First, `Triple::has_own_sysroot()` is not an mcpp +peculiarity: "the toolchain arrives with its own system" is a real axis that +the most aggressive bundler in the field also recognises. Second, the line Zig +draws — bundle what is freely redistributable, decline the vendor SDKs — is +the line this ecosystem reached independently from the licence text, and +`xim:iphoneos-sdk`'s three-tier posture is the same conclusion. + +### 2.3 CMake puts the target knowledge in the SDK, and it is why CMake cross-compilation is per-SDK folklore + +CMake has `CMAKE_SYSTEM_NAME` and toolchain files, and for these three +platforms the knowledge lives in a file the **SDK** ships: Emscripten's +`Emscripten.cmake`, the NDK's `android.toolchain.cmake`, Apple's +`CMAKE_OSX_SYSROOT`. There is no list of targets in CMake to be complete +about. + +The cost is that every SDK invents its own variables and every project learns +three unrelated dialects. mcpp's closed table is the opposite trade: adding a +platform is an engine change, and in exchange `--target
` means one +thing everywhere. The wasm row is the evidence that the trade is payable — the +row cost seven engine changes and zero project-side vocabulary. + +### 2.4 Bazel and the platform tools agree on the last row + +Bazel has platforms and toolchains in the core and registers them from +**external rulesets**: `rules_android`, `rules_apple`, `emsdk`. And the +packaging step is always the platform's own tool — Gradle produces the `.aab`, +Xcode the `.ipa`. Nobody reimplements those. + +That is the `dist-*` family's justification, and it is unanimous across every +system surveyed: **artifact shape is never core.** + +## 3. Where the current design is right + +* **A closed target table with a tier column.** Matches Rust. The alternative + (CMake) moves the cost onto every project. +* **`dist-*` in the plugins.** Matches everyone. +* **The payload in the index, with the licence deciding its tier.** Matches + Zig's line and is the only one of the four that states the reason. +* **`has_own_sysroot()` as a target property read by the shared producer.** + The wasm work found three independent readers of this question, and putting + the answer in one place is what stopped the fourth from being missed. + +## 4. Four things the design does not yet answer, and each is engine-side + +These are not defects in what shipped. They are decisions the next two rows +force, and every one of them is in the engine rather than in a plugin — which +is itself the finding: **the plugin boundary is already in the right place; +what is unfinished is upstream of it.** + +### 4.1 The iOS simulator cannot be spelled, and `Triple` has no field for it + +mcpp's model is `Triple{arch, os, env}` and the canonical form drops the +vendor: `aarch64-ios`, against Rust's and LLVM's `aarch64-apple-ios`. Dropping +the vendor is defensible and consistent (`aarch64-macos` → +`aarch64-apple-darwin` in `llvm_triple()`), and the table's own comment +defends it. + +But Rust spells the simulator `aarch64-apple-ios-sim`, and that trailing +`-sim` is a **fourth** component. mcpp's row comment says the simulator "is +deliberately not a row... folding it in would make two targets share an +identity" — which is right, and leaves the question open rather than answered: +with three fields and no vendor, there is no place for `sim` except `env`. + +`env = "sim"` would work and reads oddly, because every other `env` value +names a C library or an ABI. The alternative is a fourth field, which touches +every triple in the system. + +**iOS development without the simulator is not iOS development**, so this is +not deferrable to after the row lands. It should be decided before +`aarch64-ios` moves off `planned`, and the decision belongs in `triple.cppm`. + +### 4.2 Android's API level has nowhere to live + +The NDK's own target is `aarch64-linux-android21` — the API level is part of +the **LLVM triple**, not a flag. mcpp's row is `aarch64-linux-android` with no +version, and the level has to come from somewhere because it changes the ABI: +it selects which bionic symbols exist. + +Three places it could go, and they are not equivalent: + +* `env = "android21"` — puts it in the identity, so the output directory, + `cfg(env = ...)` and the packed ABI tag all carry it. Correct, and it + multiplies the table by every level anyone wants. +* a `[target.aarch64-linux-android] api = 24` manifest key — keeps the table + small, and the level must then enter the **fingerprint** explicitly or two + different ABIs share a build directory. That is the defect class this + codebase has recorded most often. +* the row pins one level — simple, honest, and wrong for anyone shipping to a + different minimum. + +Rust has the same problem and answers it outside the triple. Whatever mcpp +picks, **the level must be in the fingerprint**, and that is an engine change +no plugin can make. + +### 4.3 The family axis and the payload axis are conflated, and I hit it + +`emsdk` normalises to `Family::Llvm`, because `em++` *is* clang and inventing a +fourth family value would be a claim about the compiler that is false. That is +the right conclusion about the compiler and it has a consequence I ran into +directly: `mcpp toolchain list` reports the payload as `llvm@6.0.9`, which is +indistinguishable from the real `xim:llvm` in any listing keyed on +`family@version` — and the target matrix's scan takes **one toolchain per +family**, so two llvm-family payloads on one host cannot both be enumerated. + +The two axes are genuinely different questions: + + family what flag vocabulary does this compiler speak? -> llvm + payload which archive provides it? -> emsdk + +`to_xim_package` already answers the second from the target. What is missing is +that nothing *displays* the second, so a user with emsdk installed sees `llvm +6.0.9` and a matrix cell cannot name it. The fix is a display/identity field on +the resolved toolchain, not a new family — and it is engine-side. + +### 4.4 A non-MSVC toolchain cannot be located on the machine, which is the iOS tier-3 shape + +`parse_toolchain_spec` refuses `@system` for every family but MSVC, by name, +with a written reason: Visual Studio is often already installed and cannot +always be redistributed, and that is a concession to one platform rather than +a general capability. + +The iOS SDK is the second instance of exactly that situation, and +`xim:iphoneos-sdk`'s own header names "locate what the machine already has" as +the third of its three tiers. If the licence ever forces that tier — or if a +developer with Xcode installed simply prefers it — **the engine currently has +no spelling for it.** `aarch64-ios` at tier 3 is unreachable, and the refusal +that blocks it is one that argues from MSVC's uniqueness. + +That argument is now weaker by one instance. Whether to generalise `@system` +or to add a second named exception is a decision; having neither is not. + +## 5. The runner, the emulator and the simulator: where "how do I run this" belongs + +This is the part the first draft of this document only flagged. It needs +designing, because two of the three platforms cannot be exercised by executing +a file. + +### 5.1 Running an artefact has three shapes, not one + +Measured across the three rows: + +| shape | example | what it needs | +|---|---|---| +| **the artefact runs itself** | `wasm32-emscripten` | nothing. `em++` writes `#!/usr/bin/env node` and marks the file executable, and `node` on PATH is the xvm shim `xim:emsdk` already depends on | +| **a translator wraps it** | `aarch64-linux-android`, static or with `-L` | one argv prefix: `qemu-aarch64 -L ` plus an env var | +| **a session exercises it** | the Android emulator, the iOS simulator, a device | boot, transfer, execute remotely, collect the exit code and stdout, tear down | + +mcpp's `runner` key is an **argv prefix**. It covers the first two shapes +exactly and **cannot express the third**, because the third is a stateful +lifecycle rather than a command. + +### 5.2 The industry answer is unanimous, and it is cheaper than a new mechanism + +Every system surveyed provides an argv-prefix hook in the core and puts the +device lifecycle outside it: + +| system | the hook | shape | +|---|---|---| +| cargo | `target.
.runner` | program + args | +| CTest | `CMAKE_CROSSCOMPILING_EMULATOR` | program + args | +| Bazel | `--run_under` | program + args | +| Gradle / Xcode | `connectedAndroidTest`, `xcodebuild -destination` | a session, owned by the platform's own tool | + +Cargo's reference is explicit about the boundary: the runner is invoked "with +the actual executable passed as an argument", and managing devices, emulators +or simulators is **out of scope** — "that responsibility falls to the runner +program itself." + +That sentence is the design. mcpp does not need a fourth member family, a +session protocol, or any engine change: + +* `runner` stays an argv prefix. It already is, and it already matches three + other build systems. +* **The program the prefix names is an ecosystem package.** A program that + boots an AVD, pushes a binary, runs it under `adb shell`, collects the exit + code and tears down is an ordinary executable — so it is a `xim:` package + shipping a binary, exactly like `xim:qemu-user-aarch64` is today. + +So the split is: + + engine `runner` = argv prefix, and the row's default value + ecosystem the program that prefix names, including any device lifecycle + plugin nothing -- a plugin is a build-time module, and a runner is a + run-time program; putting it there would be a category error + +### 5.3 Why a `dist-*`-style plugin is the wrong home, stated so it is not tried + +A member in the `dist-*` family is a C++ module compiled into the build +program. It runs during the build and its output is a build-graph action. A +runner runs **after** the build, once per `mcpp run` or `mcpp test`, and has to +survive the build system exiting. The two have different lifetimes, and the +only thing they share is the word "platform". + +The taxonomy already says this: `rules-*` is what goes into the compile, +`tools-*` what runs beside it, `dist-*` what comes out of the link. None of +those is "how the output is exercised", and the reason is that the answer is +not a module. + +### 5.4 What each row's runner is, concretely + +| row | `runner` default | provided by | +|---|---|---| +| `wasm32-emscripten` | **none** | nothing needed -- the shebang and `xim:node` | +| `x86_64-linux-android` | none for a static artefact on an x86_64 host; a session program for the dynamic default | `xim:android-platform-tools` + `xim:android-emulator` + `xim:android-system-image` | +| `aarch64-linux-android` | `qemu-aarch64 -L ` | `xim:qemu-user-aarch64` + the extracted root `xim:android-system-image` already produces | +| `aarch64-ios` | **none possible** | an artefact cannot run off an iOS device; the simulator needs `xcrun simctl` on a macOS host | + +The aarch64 Android row is the interesting one: its runner is an argv prefix +plus one environment variable, which is exactly what the key expresses, and +every piece is already published. **It needs no new mechanism at all** — +which is why §4.2's identity decision is the only thing in front of it. + +And the x86_64 Android row is the one that wants a session program, because +its DEFAULT configuration is dynamic and the emulator is what supplies a real +`linker64`. That program does not exist yet and is a package, not a feature. + +### 5.5 The simulator is a target, not a runner, and that is the whole confusion + +An iOS simulator build is **a different target**: its own SDK, its own object, +`x86_64`/`aarch64` host-native code rather than device code. Calling it "a way +to run the iOS target" is the category error the row's own comment already +warns about -- "folding it in would make two targets share an identity". + +So the simulator needs a ROW, and §4.1's missing spelling is what blocks it. +With `env = "sim"` the two rows are `aarch64-ios` and `aarch64-ios-sim`, which +is Rust's `aarch64-apple-ios` / `aarch64-apple-ios-sim` pair modulo the vendor +elision mcpp already performs everywhere. Once the row exists, its runner is an +ordinary argv prefix over `xcrun simctl spawn`, on a macOS host, provided by a +package -- no new mechanism, again. + +## 6. Recommendations, as decisions rather than options + +| # | question | recommendation | why | +|---|---|---|---| +| R1 | the simulator's spelling | `env = "sim"`, giving `aarch64-ios-sim` and `x86_64-ios-sim` as their own rows | matches Rust's pair modulo a vendor elision mcpp already does; satisfies the row comment's own objection, which was to NOT having a separate row | +| R2 | Android's API level | **`min_api_level` under `[target.
]`, reusing the `macos_deployment_target` design** -- see 12.1 | mcpp maintains its own vocabulary and MAPS to a compiler target, so where LLVM carries the level says nothing about where mcpp stores it. macOS already does this exactly: a manifest key, a clean canonical triple, the level appended by `llvm_triple(param)` -- which already takes a version -- and the value in the fingerprint. One NDK serves a range of levels, so it is a project decision and not a toolchain property. No new rows | +| R3 | the payload identity shown for `emsdk` | a display identity on the resolved toolchain, not a fourth `Family` | `em++` is clang and a fourth family would be a false claim about the compiler; what is missing is only that nothing prints which archive answered | +| R4 | `@system` for a non-MSVC family | **withdrawn** -- see 12.1a | the second instance dissolved: `xim:iphoneos-sdk` serves iOS as a package, so no host locator is required. Generalising would admit `gcc@system`, which the existing refusal names by name and which costs hermeticity. A refusal should not be relaxed without a case | +| R5 | device and simulator sessions | a `xim:` package shipping a runner program, named by `runner` | cargo states this boundary explicitly; no engine change, no new member family, and it puts platform knowledge in the ecosystem | +| R6 | the tier each row can reach | `verified` for wasm (reached); `preview` for both Android rows and for iOS | Rust rates all three Tier 2. `verified` for Android is reachable and needs a CI lane, not a design | +| R7 | signing a Mach-O or a `.app` | package `rcodesign` as `xim:rcodesign` and have `dist-apple` prefer it | MPL-2.0 with prebuilt static binaries for linux-musl (both arches), macOS universal and Windows. It removes the last host dependency from the iOS BUILD path, leaving only a device, the Simulator runtime and a notarization credential -- none of which is a program | +| R8 | the four-field spelling | `parse()` should accept `wasm32-unknown-emscripten` and `aarch64-apple-ios` and canonicalise them | the industry writes four fields; refusing the spelling every other toolchain prints is a UX cost with no design benefit, and `parse()` already normalises several aliases | +| R9 | `mcpp pack --format ipa` | a `dist-ipa` member: zip `Payload/.app/`, after `dist-apple` and `rcodesign` | it needs NO new tool. Every other link is already in the ecosystem or one member away, so iOS PACKAGING closes entirely -- what does not close is the credential and the runtime | +| R10 | `--format dmg` and `--format pkg` | recorded as gaps with a known shape, not attempted | each needs a *creator* as well as a signer (`libdmg-hfsplus`; `xar`), both open source and neither measured here | +| R11 | the macOS rows' runner | Darling recorded as an unmeasured candidate | GPL-3.0, active, and it REIMPLEMENTS Darwin's libraries rather than redistributing them, so unlike the iOS image it carries no licence blocker. A row does not move on a plausible mechanism, so this is a candidate and not a plan | +| R12 | real-device run for both platforms | `xim:android-platform-tools` (have) and a new `xim:pymobiledevice3`, each named by a `runner` program | neither needs Apple or Google software. It supersedes the simulator route rather than complementing it: a device brings its own OS, so the only thing crossing the boundary is a signature the developer already owns | +| R13 | the iOS image | **the RUNNER PROGRAM owns the path, resolved at run time.** The plugin declares a runner by NAME; the index ships the emulator and no image; the user configures the program | three homes were considered. A locator has nothing to probe. `build.mcpp` is per-package and committed, while a path is per-MACHINE -- the same analysis #564's `default_jobs` needed -- and a path read through `env_or` is not in the program's re-run key, so changing it would appear not to change anything while the cache record persists the old one. Declaring a NAME is cache-safe; baking a PATH is not. Run-time resolution has no record to go stale | + +## 7. User-facing experience, which is the test of all of the above + +The whole point of keeping identity in the engine is that the user types one +thing: + + mcpp build --target wasm32-emscripten works today + mcpp run --target wasm32-emscripten works today + mcpp build --target aarch64-linux-android after R2 + mcpp run --target aarch64-linux-android after R2 + R5's package + mcpp build --target aarch64-ios-sim after R1 + mcpp pack --target aarch64-ios --format app after R1, R4 and dist-apple + +One verb per intent, the target as a flag, and no per-platform mode. Compare +what the alternatives ask of a user: + +* **CMake** — a different toolchain file per platform, each with its own + variable vocabulary (`EMSCRIPTEN`, `ANDROID_ABI`, `CMAKE_OSX_SYSROOT`). +* **Gradle / Xcode** — a separate task graph and project model per platform. +* **Bazel** — a ruleset per platform, loaded in `WORKSPACE`. + +mcpp's cost for that uniformity is that a platform is an engine change. The +wasm row is the first real measurement of that cost: **seven engine changes, +and zero new vocabulary for the project.** A project that builds for Linux +builds for the web by changing one flag. That is the argument for the design, +and it is now measured rather than asserted. + +## 8. Is `wasm32-emscripten` a standard name, and a common one? + +Two different questions, and the answers differ. + +### 8.1 The industry name is the four-field one, and mcpp's is a normalisation + +Measured, not recalled -- `em++ -v` on this machine passes to its own clang: + + -target wasm32-unknown-emscripten + +and `rustc`'s platform table lists the same spelling at Tier 2 with host +tools. That is the industry name. + +mcpp writes `wasm32-emscripten`, and `llvm_triple()` restores the vendor +(`triple.cppm:172` emits `arch + "-unknown-emscripten"`). So the short form is +**mcpp's canonical spelling, not a spelling anyone else uses** -- and it is the +same elision mcpp already performs everywhere: `aarch64-macos` becomes +`aarch64-apple-darwin`, `x86_64-windows-gnu` becomes `x86_64-w64-windows-gnu`. + +That is defensible and should be stated as what it is. mcpp's `Triple` has +three fields and no vendor, deliberately, and `unknown` is a placeholder that +carries no information for any target in the table. A user who types the +four-field form should still be understood, which is a `parse()` question +rather than a naming one. + +### 8.2 The wasm family is nine targets, and the three-field model holds + +`rustc`'s table lists nine: + + wasm32-unknown-emscripten Tier 2 with host tools + wasm32-unknown-unknown Tier 2 with host tools + wasm32-wasip1 Tier 2 with host tools + wasm32-wasip1-threads Tier 2 with host tools + wasm32-wasip2 Tier 2 with host tools + wasm32v1-none Tier 2 without host tools + wasm64-unknown-unknown Tier 3 + wasm32-wali-linux-musl Tier 3 + wasm32-wasip3 Tier 3 + +Mapped onto `Triple{arch, os, env}`: + +| Rust | mcpp | field use | +|---|---|---| +| `wasm32-unknown-emscripten` | `wasm32-emscripten` | os = emscripten | +| `wasm32-unknown-unknown` | `wasm32-none` | os = none, i.e. `is_freestanding()` | +| `wasm32-wasip1` | `wasm32-wasi` + env | os = wasi | +| `wasm32-wasip1-threads` | env = `p1-threads` | **env absorbs Rust's fourth component** | +| `wasm64-unknown-unknown` | `wasm64-none` | arch = wasm64 | + +The last row of that table is the important one, and it settles §4.1 from an +unexpected direction. Rust appends a fourth component for a *variant*: +`-threads` here, `-sim` for the iOS simulator. mcpp has exactly one slot for +it, `env`, and the wasm family shows that slot is adequate and already used +that way by every other row (`gnu`, `musl`, `eabihf`). So **`env = "sim"` is +not a workaround; it is the field doing its job**, and R1 is a use of the model +rather than a stretch of it. + +## 9. Is the toolchain bound to the SDK? Three platforms, two answers + +This is the question that most changes how a row is written, and the three +platforms do not agree. + +| platform | compiler | its system | one archive? | mcpp's columns | +|---|---|---|---|---| +| Emscripten | `em++` (clang) | `/emscripten/cache/sysroot` | **yes** | `pin = emsdk@6.0.9`, `sysroot` empty | +| Android | NDK's `clang++` | bionic, in `toolchains/llvm/prebuilt//sysroot` | **yes** | `pin = android-ndk@V`, `sysroot` empty | +| iOS | **any sufficiently new clang** | the iPhoneOS SDK, reached with `-isysroot` | **no** | `pin = llvm@V`, `sysroot = xim:iphoneos-sdk@V` | + +So `has_own_sysroot()` is not an arbitrary set of two: it is exactly the +platforms whose compiler and system arrive as one payload, and that is why the +predicate reads the way it does. + +**And iOS is structurally the same shape as bare metal.** `riscv64-none-elf` +pins `llvm@22.1.8` and names `xim:picolibc-riscv@1.8.12` in the `sysroot` +column -- a generic clang plus a separately-versioned system. iOS is that +shape with a different sysroot package. The `sysroot` column already exists +for precisely this, which means the iOS row needs **no new table machinery**, +only the two columns filled. + +The consequence for the other two is the opposite: a `sysroot` entry for +Emscripten or Android would be wrong, because the driver resolves its own and a +second answer competes with it -- which is the defect the wasm row hit three +times. + +### 9.1 "The Android SDK" names two unrelated things, and the packages split on that + +Worth stating because the naming misleads: + +* the **NDK** is the compiler and bionic -- one archive, bound, used at BUILD + time. `xim:android-ndk`. +* the **SDK** proper is `platform-tools` (adb, fastboot), the emulator, system + images and build-tools -- used at RUN and PACKAGE time, and unbound both from + each other and from the compiler. `xim:android-platform-tools`, + `xim:android-emulator`, `xim:android-system-image`. + +Four packages rather than one is therefore not a decomposition choice; it is +what upstream actually ships, and each is independently versioned by Google. + +## 10. What can close inside the ecosystem, and what cannot + +The preference is stated: close the loop inside xlings, and reach the host only +where nothing else is possible. Enumerated per platform rather than argued. + +### 10.1 Closed today, measured + +| need | package | evidence | +|---|---|---| +| `em++`, the wasm sysroot and its module surface | `xim:emsdk` | `mcpp run --target wasm32-emscripten` printed `1-2-3` | +| the interpreter `em++` execs | `xim:python` | needed an aarch64 payload added; declared as a dep | +| the JS engine the artefact needs | `xim:node` | the artefact's own `#!/usr/bin/env node` resolves the xvm shim | +| the Android compiler and bionic | `xim:android-ndk` | `import std` built for both Android arches | +| an aarch64 loader and a real bionic | `xim:android-system-image` | `debugfs` extraction, then `qemu-aarch64 -L` ran the DYNAMIC artefact | +| the user-mode translator | `xim:qemu-user-aarch64` | same measurement | +| the ext4 reader that extraction needs | `xim:e2fsprogs` | declared; was a host probe | +| `adb` / `fastboot` | `xim:android-platform-tools` | installs on all three hosts | +| the emulator and its X11 chain | `xim:android-emulator` + six existing libs | declared; was a host probe | + +### 10.2 Closeable, and one of them is a finding + +| need | how | status | +|---|---|---| +| a clang that targets iOS | `xim:llvm` plus `-isysroot` | the payload exists; the row is unfilled | +| the iPhoneOS SDK | `xim:iphoneos-sdk`, at whichever of three licence tiers applies | exists | +| **signing a Mach-O, a `.app`, a `.dmg` or a `.pkg`** | **`rcodesign`** (crate `apple-codesign`, MPL-2.0) | **not yet packaged, and it should be** | +| finding the SDK path | nothing -- `xcrun` is a path-finder and `-isysroot ` needs none | no dependency | + +The third row changes the iOS picture. `apple-codesign` states its goal as +being "a stand-in replacement for Apple's `codesign` ... without a dependency +on an Apple hardware device or operating system", covering Mach-O binaries, +`.app` bundles, `.pkg` installers and `.dmg` images. Release 0.29.0 ships +**prebuilt static binaries for `x86_64-unknown-linux-musl`, +`aarch64-unknown-linux-musl`, macOS universal and Windows**, under MPL-2.0. + +So signing -- which `dist/apple.cppm` currently reaches through the host's +`codesign` -- **is not a host dependency at all**. It is an unpackaged one. + +Sources: [apple-codesign on crates.io](https://crates.io/crates/apple-codesign) +and its [documentation](https://gregoryszorc.com/docs/apple-codesign/stable/). + +### 10.3 Signing belongs in the ecosystem AND in the plugin system, in that order + +`rcodesign` is a program, so it is a `xim:` package; what it is invoked BY is a +`dist-*` member; and what the user types is `mcpp pack --format `. All +three layers already exist, which is why this needs no new mechanism -- only +the package and the members. + + xim:rcodesign the program (MPL-2.0, prebuilt static) + dist-apple --format app the bundle exists, uses the host's codesign today + dist-ipa --format ipa the shippable file does not exist + dist-dmg --format dmg a disk image does not exist + dist-pkg --format pkg an installer does not exist + +Release 0.29.0 signs **bundles**, not only flat Mach-O binaries -- its +changelog discusses `--shallow` bundle mode and child-bundle signing "compatible +with the behavior of Apple's `codesign`" -- which is exactly what a `.app` +inside an `.ipa` needs. + +**`--format ipa` is the one that needs no new tool at all.** An `.ipa` is a zip +containing `Payload/.app/`, so the chain is: clang plus the iPhoneOS SDK +produce the Mach-O (both `xim:`), `dist-apple` assembles the bundle, `rcodesign` +signs it, and a zip step produces the file. Every link is already in the +ecosystem or is one member away. + +`--format dmg` and `--format pkg` each need a *creator* as well as a signer, +and neither creator is packaged: a `.dmg` is an HFS+/APFS image (Apple's +`hdiutil`, or `libdmg-hfsplus` off macOS) and a `.pkg` is an XAR archive +(Apple's `pkgbuild`, or `xar`). Both alternatives are open source and neither +has been measured here, so they are named as gaps with a known shape rather +than claimed. + +### 10.4 The iOS runtime: the blocker is a licensed IMAGE, not a missing emulator + +This is the question worth getting exactly right, because the obvious answer is +wrong in an instructive way. + +QEMU can emulate ARM iOS hardware, and community projects exist that boot iOS +on it. What none of them can supply is the **iOS kernel and root filesystem**: +distributing iOS images is against Apple's terms, so an image must be one the +user already legally owns. The emulator is not the scarce thing. + +That is the same shape as the Android question, with the opposite answer, and +the comparison is the point: + +| | the emulator | the OS image | can the loop close? | +|---|---|---|---| +| Android | Apache-2.0, and `emulator/LICENSE` says so | AOSP `default` builds, OSS notices throughout | **yes** -- both are packaged, and `qemu-aarch64 -L` needs no emulator at all | +| iOS | QEMU, GPL, packageable | **not redistributable** | **no** -- and no amount of tooling changes it | + +So `aarch64-ios` cannot reach `verified` for a reason that is not about mcpp, +xlings, or effort. It is the one row in the table whose execution is blocked by +a licence rather than by work, and saying so precisely is better than leaving +it as "needs a device". + +Sources: [iOS emulators, Emulation General Wiki](https://emulation.gametechwiki.com/index.php/IOS_emulators) +and [Emulating iOS on Linux](https://linuxvox.com/blog/emulate-ios-on-linux/). + +**On macOS, the Simulator is the host's and that is fine.** It ships with Xcode, +it is a (b)-category proprietary runtime, and `xcrun simctl spawn` is an argv +prefix -- so §5's model covers it with no new mechanism, on a macOS host, once +R1 gives the simulator a row. + +### 10.5 Real devices close for BOTH platforms, and that is a better answer than an emulator + +Asked directly, and it turns out to be the strongest result in this section: +**running on real hardware needs no Apple or Google software on either +platform.** + +| platform | what installs and launches | licence | state | +|---|---|---|---| +| Android | `adb push` + `adb shell` | Apache-2.0 | **already packaged** -- `xim:android-platform-tools`, installs on all three hosts | +| iOS | [`pymobiledevice3`](https://github.com/doronz88/pymobiledevice3) | GPL-3.0 | pure Python 3, no compiled extensions, Linux / Windows / macOS; 2736 stars, last push 2026年09月10日 | +| iOS | [`libimobiledevice`](https://github.com/libimobiledevice/libimobiledevice) | LGPL-2.1 | the C library it was modelled on; 8177 stars, last push 2026年06月10日 | + +`pymobiledevice3` describes itself as requiring no Xcode, working with the +system `usbmuxd`, and covering app management plus iOS 17+ developer tooling +over a tunnel. `libimobiledevice` states it needs no jailbreak. Neither +requires a Mac. + +So the iOS row's execution story is not "needs a device on a Mac". It is: + + build xim:llvm + xim:iphoneos-sdk closed + bundle dist-apple (--format app) closed + sign xim:rcodesign closeable, R7 + package dist-ipa (--format ipa) closeable, R9 + deploy+run xim:pymobiledevice3 closeable, R12 + ------------------------------------------------------------------------ + entitlement a provisioning profile and a signing identity NOT closeable + +**Only the last line does not close, and it is a credential rather than a +tool.** Installing on a non-jailbroken device requires an Apple Developer +provisioning profile; `pymobiledevice3` can install a signed `.ipa` and cannot +conjure the entitlement. That is category (c), and no package manager closes a +credential -- which is the same boundary a developer already lives with when +using Xcode. + +That is a materially better position than the simulator route, and it is worth +stating why: the simulator is blocked by a **licensed image** that cannot be +redistributed, while a real device supplies its own OS and the only thing +crossing the boundary is a signature the developer already owns. + +### 10.6 Where a device session lives: the runner program absorbs deployment + +§5 concluded that a device session is a runner PROGRAM in a `xim:` package, +named by an argv-prefix `runner`. Deployment does not need a fourth verb, and +the reason is that `adb push && adb shell` is one operation from mcpp's side: + + mcpp run --target aarch64-linux-android + -> runner = ["mcpp-android-device-run"] a xim package's program + which pushes, executes, collects the exit code and stdout, tears down + + mcpp run --target aarch64-ios + -> runner = ["mcpp-ios-device-run"] a xim package's program + which installs the signed .ipa, launches it, streams the log, collects + +So both mechanisms are used, each for what it is: + + plugin produces the artefact dist-ipa, dist-apple (build time) + package deploys and runs it the runner program (run time) + engine names the runner the `runner` key (already exists) + +And the ordering is a real dependency rather than a convention: the iOS device +runner has nothing to install until `dist-ipa` has produced a signed file, so +R9 precedes R12. + +### 10.7 The iOS image: the ecosystem supplies the PROGRAM, the user supplies the BYTES + +The principle is not in question: an image in a public index is redistribution +of Apple's operating system whatever it is labelled, and a "temporary, disabled +later" flag does not change it -- anyone resolving the index installs it. This +differs from the Android decision earlier in this document in a way worth +stating precisely: there, Apache-2.0 licence files were verified INSIDE the +archives and clause 3.5 genuinely applies; here there is no open-source +component to invoke. + +What took analysis is WHERE the path lives. Three candidates were considered +and two are wrong for reasons worth recording, because each looked right first. + +#### Rejected: a locator package + +A locator works when the thing has a CONVENTIONAL location to probe -- +`vswhere` for Visual Studio, `/Applications/Xcode.app` for Xcode. An image a +user legally owns is wherever they put it, so a locator has nothing to probe +and would be a package whose entire content is a question. Its version axis +would be meaningless too: a locator for `iphoneos-image@18.0` cannot verify +that what it found is 18.0. + +#### Rejected: the path in `build.mcpp`, and the reason is this repository's own + +A build program is per-package, committed to a repository, and its declarations +are persisted in the build cache record. An image path is none of those things: + + it is per-MACHINE two developers keep it in different places + it is not committable an absolute path in someone's home directory + it must not be a build input identical sources must not produce different + build directories because a path differs + +That is precisely the analysis `[build] default_jobs` needed (#564): the +precedence is invocation> project> **machine**, and an image path sits on the +machine level exactly as a job count does. Putting a machine fact in a +per-package file is the shape that key was fixed for. + +And there is a sharper failure. If a build program reads +`MCPP_IOS_IMAGE_ROOT` through `env_or`, that variable is **not** part of +mcpp's contract environment, so it is not in the program's re-run key. Change +the path and the program does not re-run; the stale runner replays from the +cache record, which persists it (tag `"runner"`). The result is a path that was +changed and appears not to have been -- the defect class this repository has +recorded most often, and here it would be introduced deliberately. + +#### The design: the runner PROGRAM owns the path, and nothing above it knows + + engine names a runner. Does not know what an image is. + index ships the emulator (QEMU is packageable) and NO image. + plugin declares WHICH runner, and produces the artefact to run. + runner a program in a xim package. Owns the path, at RUN time. + user keeps the bytes, and tells the runner program where they are. + +The path enters mcpp at no point: not the index, not the build program, not the +cache record, not the fingerprint. What is published is a program that takes a +path, and the one thing crossing the boundary is the user's own configuration +of that program. + +This is the boundary cargo states for itself, quoted earlier in §5.2: managing +devices and simulators is out of scope, and "that responsibility falls to the +runner program itself." A runner program that owns its own configuration is the +same sentence applied one level further. + +It also dissolves the staleness problem rather than mitigating it. A run-time +resolution cannot be stale, because there is no record to go stale -- the +program reads its configuration each time it is invoked, which is what a +machine fact wants. + +#### What each layer actually writes + +The plugin side declares the runner by name, so it is available and not +imposed: + + // a dist/run member, or the project's own build.mcpp + mcpp::runner("device", "mcpp-ios-device-run"); // needs no image at all + mcpp::runner("qemu", "mcpp-ios-qemu-run"); // reads its own config + +reached as `mcpp run --runner device` or `--runner qemu`. Two properties of the +existing machinery make this work unmodified: the `runner` directive's cache +tag is non-empty so a declaration survives a cache hit, and its +`Scope::RunGlobal` is correct because a runner is a property of the invocation +rather than of one package in the graph. **Declaring a NAME is cache-safe; +baking a PATH is not** -- which is the whole distinction this section arrived +at. + +#### This is a pattern, and naming it is worth more than the iOS instance + +The same contract serves every "you have it, we cannot ship it" case: a vendor +BSP under NDA, a licensed board-support blob, proprietary firmware, a paid SDK. +In each, the ecosystem packages the TOOL that consumes the bytes and never the +bytes, and the tool owns its own configuration. + +Stating it as a pattern matters because the alternative -- deciding case by +case -- is how a "temporary" entry becomes permanent. And it composes with R12 +in the direction that counts: **with a real device supported, no image is on +the critical path at all.** The image route serves a developer who has one and +prefers it; the row does not depend on it. + +### 10.8 Darling is a candidate for the macOS rows, and is recorded as unmeasured + +[Darling](https://github.com/darlinghq/darling) is a macOS compatibility layer +for Linux -- GPL-3.0, actively developed (last push 2026年09月06日). It reimplements +Darwin's system libraries rather than redistributing them, so it carries **no +Apple licence blocker**, which makes it categorically different from the iOS +image problem above. + +It runs macOS binaries, not iOS ones, so it is irrelevant to `aarch64-ios` and +potentially relevant to `x86_64-macos` and `aarch64-macos` -- the first of which +is `planned` in this table with no host able to serve it off an Apple machine. + +Recorded as a candidate and explicitly **not** as a plan: nothing in this +ecosystem has run it, its coverage is partial by construction, and a row does +not move on a plausible mechanism. What it would be, if it worked, is an +ordinary `runner` argv prefix supplied by a `xim:` package -- the same shape as +`qemu-user-aarch64`. + +### 10.9 Genuinely host-bound, and there are exactly three + +1. **`/dev/kvm`** -- a kernel facility. No package ships a kernel feature, and + group membership is a machine's configuration. This is why it is the only + `log.warn` left in `android-emulator.lua`. +2. **A real device, Apple's Simulator runtime, or a licensed OS image.** The + simulator is a proprietary runtime that exists only on macOS; a device is + hardware; and an iOS kernel plus root filesystem cannot be redistributed at + all. Note which of the three is the actual blocker for emulation: QEMU is + packageable and the IMAGE is not, which is precisely why the same mechanism + closes for Android -- where the image is AOSP -- and cannot for iOS. +3. **Notarization.** Apple's servers plus a developer credential. A credential + is never a package, and `rcodesign` can drive the submission but cannot + supply the account. + +### 10.10 The rule that falls out + + A host dependency is legitimate only when the thing needed is + (a) a kernel facility, + (b) a proprietary RUNTIME that exists only on its own OS, or + (c) a credential. + Anything that is a PROGRAM can be packaged, and the survey found that + even Apple's signing tool has a redistributable replacement. + +That test is worth having because it is falsifiable, and it immediately +reclassifies two things this ecosystem had treated as host dependencies: +`debugfs` and the libX11 chain were (a)-shaped in the recipes' prose and were +in fact just programs. `codesign` is the third instance of the same mistake, +and this document is the first place it is named. + +It also narrows R4. `msvc@system` is a (b): Visual Studio is a proprietary +toolchain that exists only where it is installed. Generalising `@system` +should therefore mean "a row may declare that its system is host-located +because it is (b)", not "any family may be located on the host" -- the +existing refusal is right about the general case and wrong only about +believing MSVC is the sole instance. + +## 11. The rule, stated so the next platform does not need this document + +Three questions, and the answer to each is the same for every platform: + +1. **What is this target?** — engine. A package cannot add a row, because + every layer above attaches to one: the output directory, `cfg()`, the ABI + tag, the fingerprint, the runner. Rust and Zig both keep this closed and in + the core. + +2. **Which compiler serves it, where does it live, and what must it be told?** + — engine, because these are properties of the **target**, not of any + package. The wasm row needed seven such answers and every one of them was a + predicate that already existed and was right about the rows its author had + in mind. A plugin cannot fix `resolve_link_model`. + +3. **What file does the user ship?** — plugin. Unanimous across CMake, Bazel, + Gradle, Xcode and Emscripten: the format's own tool owns the format. + +And one corollary, which is the boundary's real test: **if a question's answer +differs per project, it is a manifest key; if it differs per target, it is +engine; if it differs per artifact shape, it is a plugin.** The Android API +level is the interesting case, because it looks like the first and behaves like +the second. + +## 6. What this implies for the three rows + +| row | engine work remaining | plugin work | tier it can honestly reach | +|---|---|---|---| +| `wasm32-emscripten` | the `.wasm` sibling as an implicit link output | an `.html` shell, `--preload-file` data staging, a dev server | **`verified`** -- reached | +| `x86_64-linux-android` | the API-level decision (§4.2) | `.apk` / `.aab` assembly, which is Gradle's tool | `preview`, `verified` if a CI lane runs the emulator | +| `aarch64-linux-android` | the same, plus nothing else -- route A executes it | the same | `preview`; `verified` needs the qemu-user runner wired | +| `aarch64-ios` | the simulator spelling (§4.1) and possibly `@system` (§4.4) | `.app` (exists) and `.ipa`, which is Xcode's tool | `preview` -- there is no way to run an iOS artefact off a device | + +The two Android rows are the nearest, and the thing blocking them is not +payload work — both payloads are published and both execution routes are +measured. It is one identity decision. + +## 12. Self-review of this proposal, before implementing any of it + +### 12.1 R2, three times, and the design mcpp already has + +This recommendation was written one way, reversed on a measurement, and then +reversed back when the measurement turned out to answer a different question. +The sequence is recorded because the mistake in the middle is instructive. + +**First answer: a manifest key.** Because the API level is a per-project +minimum -- what Gradle calls `minSdk` -- and because every system surveyed +keeps it out of the triple. + +**The reversal, and why it was wrong.** A real clang was asked: + + clang -target aarch64-linux-android21 -print-effective-triple + -> aarch64-unknown-linux-android21 + +LLVM puts the level in the **env** field, so `env = "android"` looked like +LLVM's own model rather than an invention, with `llvm_triple()` staying pure. + +That measurement is correct and it settles nothing here, because **mcpp +maintains its own target vocabulary and MAPS it to a compiler target.** Where +LLVM carries the level is a fact about the EFFECTIVE triple. Where mcpp carries +it is a question about the CANONICAL one, and the two are deliberately +different -- as `prepare.cppm` says in as many words: "The triple is mcpp's +vocabulary (`aarch64-macos`); the flag carries the spelling a compiler takes +(`arm64-apple-macos14.0`)." + +The general lesson: **a measurement of another tool's model does not settle a +question about ours.** It told me where LLVM writes the level, and I read it as +telling me where mcpp should store it. + +**The design mcpp already has, and which R2 should reuse.** macOS solved this +exact problem and the machinery is complete on all three counts: + +| | macOS, today | Android, proposed | +|---|---|---| +| manifest key | `macos_deployment_target = "14.0"` in `[package]` | `api = 24` in `[target.
]` | +| canonical triple | `aarch64-macos` -- clean | `aarch64-linux-android` -- clean | +| effective target | `arm64-apple-macos14.0`, composed by `llvm_triple(param)` | `aarch64-unknown-linux-android24`, same call | +| fingerprint | `put(s, "macos", b.macosDeploymentTarget)` | the same, one line | + +`llvm_triple()` **already takes a version parameter** -- it is called as +`want->llvm_triple(macos::deployment_target(...))` -- so the objection that a +manifest key would give it a second input was already false when I raised it. +The function is not pure of versions today; it is pure of the *manifest*, which +is the property that matters, and the caller supplies the value. + +**And one NDK serves a range of levels**, so the level is not a property of the +toolchain either: naming `android-ndk@30.0.16248370` does not pin API 24. It is +a project decision, which is what a manifest key is for. + +So R2 is: **`api` under `[target.
]`, appended to the effective triple +by the existing parameter, and entered into the fingerprint the way +`macos_deployment_target` already is.** No new rows, no new mechanism, and the +table does not multiply as levels are added. + +#### The field name, chosen against Android's own vocabulary + +`api = 24` was the first spelling and it is too vague: it says nothing about +WHICH property of the API is meant, and mcpp has no other `api` key to anchor +the reading. The naming convention to follow is `macos_deployment_target`'s -- +**named in the platform's own words** -- so the question is what Android calls +this. + +Read from the NDK's own documentation rather than recalled: + +| source | spelling | what the docs say | +|---|---|---| +| NDK CMake toolchain | `ANDROID_PLATFORM` | "specifies the **minimum API level** supported by the application or library" | +| the same, alias | `ANDROID_NATIVE_API_LEVEL` | "Alias for `ANDROID_PLATFORM`" | +| Android.mk | `TARGET_PLATFORM` | "The Android **API level** number the build system is targeting" | +| Gradle | `minSdk` | the NDK docs state `ANDROID_PLATFORM` "corresponds to the application's `minSdkVersion`" | + +So Android's concept name is **"API level"** -- the term its documentation uses +most -- and the specific quantity here is the **minimum**. + +Judged against that: + +| candidate | verdict | +|---|---| +| `api` | rejected. Says nothing about which property, and anchors to nothing | +| `ndk_api_version` | rejected on two counts. "version" is not Android's word, which is "level"; and `ndk_` names the TOOLCHAIN, while one NDK serves a RANGE of levels -- so naming it after the NDK reintroduces exactly the confusion 12.1 resolved | +| `platform` | rejected. It is the NDK's own variable name, and `platform` is badly overloaded in mcpp -- a module, and the `xpm` platform tables | +| `min_sdk_version` | rejected. Gradle's `minSdk` is an application-manifest concept for the Java side; for native code the NDK's word is API level, and mcpp is not building an app | +| **`min_api_level`** | **chosen.** "API level" is Android's own term; "min" states the semantics the NDK docs state themselves; no platform prefix, because `[target.aarch64-linux-android]` already supplies it | + +The kinship with `macos_deployment_target` is worth stating: both answer "the +oldest OS release this artifact must run on", and both are named in their +platform's vocabulary rather than in a shared abstraction. A single +`min_os_version` for both would be more uniform and would cost the existing +key a rename and both platforms their own words -- which is the trade this +codebase has consistently declined. + +#### The usage model + + # mcpp.toml + [package] + name = "app" + version = "0.1.0" + + # The minimum Android API level this project supports -- the same decision + # Gradle spells `minSdk`. One NDK serves a range, so this is the project's + # to make and not the toolchain's. + [target.aarch64-linux-android] + min_api_level = 24 + +and what each layer then sees: + + mcpp build --target aarch64-linux-android + + canonical triple aarch64-linux-android identity: output directory, + cfg(env = "android"), ABI tag + effective target aarch64-unknown-linux-android24 what clang is given + fingerprint includes 24 so 21 and 24 are two build + directories, never one + + # unset is legal and means the NDK's own default, which is what + # `clang -target aarch64-linux-android` normalises to. + +The parallel with the macOS key is exact, down to `[package]` versus +`[target.
]` being the only difference -- and that difference is right: +a deployment target applies to every Apple artifact a project produces, while +an API level applies to one target row. + +### 12.1a R4 is withdrawn, because its second instance dissolved + +R4 proposed generalising `@system` beyond MSVC, on the grounds that the iOS SDK +is a second instance of "a proprietary thing that only exists where it is +installed". + +After the rest of this document, that is no longer true. `xim:iphoneos-sdk` +exists and the licence permits at least the fetch-upstream tier, so iOS is +served by a PACKAGE and needs no host locator. The locator tier its header +documents is a fallback that nothing currently requires. + +And the risk is concrete rather than theoretical: generalising the spelling +admits `gcc@system`, which the existing refusal names and refuses by name, and +which would let a build use the host's compiler and silently lose hermeticity +-- the property the whole payload model exists for. + +So the honest conclusion is not "defer until the narrow form is designed". It +is that **the motivating case evaporated, and a refusal should not be relaxed +without one.** If a real instance appears, the narrow form -- a per-row +permission defaulting to denied -- is the shape to design then. + +### 12.2 Two recommendations should be split by what they cost to be wrong about + +R1 (`env = "sim"`) and R8 (accept four-field spellings) are both cheap and +reversible: a new row is additive, and widening a parser is additive. They can +go in without further argument. + +R4 (generalise `@system`) is neither. It removes a refusal whose comment argues +at length for why it exists, and a wrong generalisation admits +`gcc@system` -- the exact spelling that comment refuses by name. The safe form +is narrow: a per-row permission, defaulting to denied, so the refusal's +reasoning stays true for every row that has not opted in. + +### 12.3 What this proposal does not measure, stated plainly + +* `rcodesign` has not been run. Its capabilities are quoted from its own + changelog and documentation. Signing a real `.app` and having macOS accept + it is the criterion, and no macOS machine has been involved. +* `pymobiledevice3` has not been run, and there is no iOS device here. +* Darling has not been run. +* The macOS and Windows legs of every payload completed today are declared + from verified hashes and have not been executed. + +Every one of those is a claim about somebody else's software, and this session's +record is that **every wrong guess in it was about what a vendor had done, and +every one was cheap to check and was not checked.** The four above are the +places that pattern would recur. + +### 12.4 The task list, so nothing is left half-done + +Grouped by repository, because the one-PR-per-repo rule makes the grouping the +plan. + +**mcpp (one PR, 2026年9月11日.3)** -- the general capability, all of it: + +| # | task | state | +|---|---|---| +| E1 | `wasm32-emscripten` resolves, builds and runs | **done**, measured `1-2-3` | +| E2 | the seven gates the wasm row needed | **done** | +| E3 | `Format::Wasm` and its mechanism | **done** | +| E4 | wasm is a capability pin | **done** | +| E5 | matrix expectations for 12 wasm cells | **done** | +| E6 | unit tests for E1-E4 | **done** | +| E7 | the EOL debian leg swapped for debian-12 | **done** | +| E8 | R8: `parse()` accepts `wasm32-unknown-emscripten`, `aarch64-apple-ios` | **done** -- vendor segments (`unknown`/`pc`/`w64`/`apple`) skipped, `ios`/`iphoneos` and `emscripten` accepted as OS segments | +| E9 | R1: `aarch64-ios-sim` and `x86_64-ios-sim` rows | **done**, `planned` -- the simulator is a target, so it gets its own identity rather than being folded into `aarch64-ios` | +| E10 | R3: a payload display identity, so emsdk is not shown as `llvm` | **done**, `ToolchainSpec::payloadName` | +| E11 | the `.wasm` sibling as an implicit link output | **not attempted** -- recorded in §11 of the decomposition doc; the artifact is produced and found, only the graph does not name it | +| E12 | R6: Android rows to `preview`; iOS stays `planned` | **done** -- `android-ndk@30.0.16248370` pin, measured below. iOS stays `planned` because its blocker is a licence, not a payload | +| E13 | docs: `04-mcpp-toml` §2.7.3, `20-toolchains`, `21-the-target-triple` + zh | **done** | +| E14 | CHANGELOG | **done** | +| E15 | R2: `min_api_level` under `[target.
]`, via `llvm_triple(param)` and the fingerprint, per §12.1 | **done** | +| E16 | `host_can_serve` stops hardcoding Linux for `has_own_sysroot()` rows | **done** -- see below; this is the line the predicate's own comment named as its expiry | + +**E16 leaves one thing unanswered, and it is recorded rather than hidden.** +With the host constant gone, `mcpp toolchain list` reports +`aarch64-linux-android` as `available` on every host -- including Windows, +where `xim:android-ndk` deliberately has no table. Google publishes a Windows +NDK and it downloads; what it does not contain is the libc++ module surface +(measured: 9108 entries, no `std.cppm`, no `std/*.inc`, against darwin's 10024 +and 110), so for a module-first build tool that payload cannot serve and an +entry that can never serve is worse than none. + +The engine is not the place to encode that. Which hosts an index serves is the +index's answer, it changes without an engine release, and a constant stating it +here is exactly what E16 removed -- it would go stale again the first time a +future NDK ships the surface. So a Windows user sees the row, the pin resolves, +and xim refuses with `no payload for this platform` before anything is fetched, +naming the package. That is legible at the point of use, which is the standard +this repository already applies to a per-package engine floor. + +What would change the answer is a cheap way to ask the index for a payload's +platform coverage during a listing. There is none today that does not cost a +network round trip per row, and a fourth status word would describe the gap +rather than close it. + +**E16 was not in the original plan, and the target matrix is what produced +it.** The predicate returned `mcpp::platform::is_linux` for a row whose SDK +ships its own sysroot, because `xim:emsdk` and `xim:android-ndk` declared only +`xpm.linux` when it was written. X2 made that false, and the two halves of one +goal then disagreed: the index published the payload on three hosts while the +engine deleted the row from `toolchain list` on two of them. The symptom was +not a wrong answer but an ABSENT one -- `mcpp build --target wasm32-emscripten` +on macOS reported a target this table knows as one it had never heard of, which +is the same defect the `planned` tier exists to avoid. + +Neither half is where it was found. `scan (macos-arm64)` and +`scan (windows-x86_64)` failed on a cell count -- 24 measured against 25 +declared -- and the one missing cell named the row. A per-host job comparing +against a checked-in table is the only thing in this repository that can see a +row disappear, because every other check asks about a row it already has. + +R4 is not in that list because it is **withdrawn** (§12.1a), not deferred: its +motivating case dissolved once `xim:iphoneos-sdk` covered iOS, and relaxing a +refusal without a case is how `gcc@system` gets in. + +**xim-pkgindex** -- one PR, already open as #812 plus the platform completion: + +| # | task | state | +|---|---|---| +| X1 | Android CN mirrors under clause 3.5 | **done**, #812 | +| X2 | emsdk, NDK and emulator on all three hosts | **done**, hashes verified | +| X3 | `xim:python` aarch64 and a GLOBAL url | **done** | +| X3a | the two host assumptions X2 introduced, found by the per-host install jobs | **done** -- the NDK's release-directory pattern named `-linux` only; emsdk's Windows entry points are `.exe`, not the `.bat` the first version guessed | +| X4 | R7: `xim:rcodesign` | todo | +| X5 | R12: `xim:pymobiledevice3` | todo | +| X6 | R5: the two runner programs | todo, and they are new software rather than packaging | + +**mcpp-plugins** -- one PR after the engine release: + +| # | task | state | +|---|---|---| +| P1 | `dist-*` family, three members | **done**, 0.6.0 tagged | +| P2 | R9: `dist-ipa` | todo | +| P3 | R7's plugin half: prefer `xim:rcodesign` over the host's codesign | todo | + +**mcpp-index**: publish `mcpp:plugins@0.6.0`. **done** -- tag published, the +GitCode asset verified by download-back and byte-identical to the GitHub source +archive, so one sha256 names both hosts; PR open. + +**Recorded and not attempted**: R10 (`dmg`/`pkg` creators), R11 (Darling), +R13's implementation (the pattern is documented; the runner programs are X6). + +### 12.5 The dependency order, and the one place it is not obvious + + mcpp engine ──► release ──► mcpp-plugins ──► mcpp-index + │ │ + └──► xim payloads ──────────┘ + (independent, merge first) + +The non-obvious edge is **X6 before P2 is wrong**. A device runner has nothing +to install until `dist-ipa` exists, so P2 precedes X6 -- the reverse of the +usual "payloads first" rule, and the reason is that here the payload consumes +the plugin's output rather than feeding it. diff --git a/.agents/docs/2026-09-11-six-open-issues-analysis.md b/.agents/docs/2026-09-11-six-open-issues-analysis.md new file mode 100644 index 000000000..049a6d133 --- /dev/null +++ b/.agents/docs/2026-09-11-six-open-issues-analysis.md @@ -0,0 +1,649 @@ +--- +subject: triage +status: active +--- + +# Six open issues: what each one actually is, and what would answer it + +Issues #564, #597, #599, #603, #604 and #606, read against the code and, where +a claim was checkable, measured rather than accepted. Every report is accurate +about its symptom; three of them are wrong about the cause, and two of those +three describe a defect that is smaller than the one that is present. + +Line numbers are against `fix/staging-is-a-service-for-a-dispatched-format` +(`c16a5128`), which is #607 on top of #605. + +## 0. What the six have in common + +Three of them -- #606, #604, #599 -- are one class of defect: **a rule applied +to an object it was not written for.** + + #606 a keyword matcher written for source lines, applied to a comment line + #604 a de-duplicator written for a one-token marker, applied to a two-token pair + #599 a hub path written for the current tree, applied to a historical one + +None of the three is a mistake in the rule. Each rule is correct about the +object it was written for, and each acquired a second object without acquiring +a second reading. That is why none of them was found by a test: a test written +alongside the rule tests the rule against the object its author had in mind. + +Two of the three -- #606 and #599 -- have a second property in common, which +is why they lasted: **the wrong answer is not loud.** #606's reported form is +an error, but its unreported form corrupts the module graph in silence; #599's +check prints a note, and a note never turns a job red. + +## 1. #606 -- the scanner reads inside block comments + +### The report is right about the symptom and wrong about the cause + +Reproduced verbatim at 2026年9月10日.2: + + /* + module (exe) + */ + int main() { return 0; } + + error: scanner errors: + src/main.cpp:2: '(exe)' is not a module name. ... + +The report bisects this to "lands after 2026年9月7日.1" and calls it a regression. +Measured: `git log -S` over `src/modgraph/scanner.cppm` returns **no commit at +all** that ever added block-comment state. The line loop +(`src/modgraph/scanner.cppm:739-751`) tracks exactly two things -- preprocessor +depth and multi-line raw strings -- and its comment filter is + + std::string_view strip_line_comment(std::string_view s) { + auto p = s.find("//"); + ... + } + +which handles `//` and nothing else. What landed at 2026年9月9日.1 (`7a4b8391`, +#594) is the `is_well_formed_module_name` refusal, and that refusal is +**correct**: a malformed name recorded into the graph becomes a BMI path that +nothing reports. The bisect therefore dates the moment the defect became +audible, not the moment it was introduced. + +### The root cause is an argument written down as settled + +The reasoning that admits the defect is in the source, stated as a completed +argument (`scanner.cppm:702-707`): + +> Ordinary `"..."` strings are intentionally left as-is: the import/module +> matcher only fires on lines whose trimmed text *starts with* the keyword, +> which a string body can only do when it spans lines (i.e. a raw string). + +The premise is sound and the enumeration is short by one. Two constructs can +put a keyword at the start of a line without it being code: a raw string, and a +**block comment**. The code handles the one the comment names. + +This is also why the report's table looks arbitrary. `/* module (exe) */` on +one line is fine because the trimmed line starts with `/*`; the same text with +the opener on its own line is not, because then the trimmed line *is* +`module (exe)`. + +### The unreported form is worse, and was measured + + /* + export module y; + */ + int main() { return 0; } + +builds successfully, and the graph it generates says: + + build obj/main.o | gcm.cache/y.gcm : cxx_object .../src/main.cpp | obj/main.cpp.ddi.dd + bmi_out = gcm.cache/y.gcm + +A plain `main.cpp` is recorded as the **producer of module `y`**, and the BMI +it promises is never written -- `gcm.cache/` is empty after a successful build. +Add a file that legitimately imports `y` and the diagnostic is: + + y: error: failed to read compiled module: No such file or directory + y: note: compiled module file is 'gcm.cache/y.gcm' + y: note: imports must be built before being imported + +The import was satisfied *from a comment*, so the real provider is never +searched for, and the message names an ordering problem that does not exist. +The scanner's own comment two arms above predicts this shape exactly: "a +recorded non-name propagates into the build graph as a BMI path and is reported +by nothing". It is a well-formed name here, so the guard that catches the +report's case does not fire. + +`import foo;` and `module x` inside a block comment are the same defect at +lower cost: a false edge and a false implementation-unit identity. + +### What answers it + +Block-comment state in the line loop, in the same shape as the raw-string +state that is already there: a `bool in_block` carried across iterations, with +the stripping done before `strip_line_comment` so that `/* */ import x;` still +scans and `// /*` does not open a block. Nesting is not a C++ construct and +must not be implemented; `/*` inside a string literal is the one remaining +corner, and the existing raw-string pass already blanks the case that can span +lines. + +The comment quoted above must be corrected in the same change. Leaving it +would leave the argument that produced the defect standing next to the code +that fixes it. + +Criterion, and it has to be the silent form, because the loud one is a +side effect of a guard that could legitimately be relaxed: + +* a `.cpp` whose only `export module y;` is inside a block comment generates a + graph with **no** `gcm.cache/y.gcm` output and no `bmi_out`, and +* the four-line file from the report builds, and +* `/* */ import x;` on one line still records the import -- the strip must + remove comment *text*, not the whole line. + +The second and third are the pair that distinguishes a fix from a mute. + +## 2. #604 -- a two-token switch loses its switch + +### Cause, read rather than guessed + +Host-module use flags are collected in `src/build/build_program.cppm:1259-1266`: + + for (auto& f : hm->useFlags) { + // GCC's marker is just `-fmodules`, already present when the + // bundled module was built; repeating it is harmless but noisy. + if (std::find(moduleFlags.begin(), moduleFlags.end(), f) + == moduleFlags.end()) + moduleFlags.push_back(f); + } + +The de-duplication is **per token**. What the three families put in `useFlags` +is not the same shape: + +| family | `useFlags` | tokens | +|---|---|---| +| GCC | `-fmodules` | 1, idempotent | +| Clang | `-fmodule-file==` | 1, unique per module | +| MSVC | `/reference`, `=` | **2, first one repeats** | + +`bmi_reference_tokens` (`src/toolchain/hostflags.cppm:296`) splits at the +prefix's last space, so `" /reference huxerui.rules.sources="` becomes exactly +`{"/reference", "huxerui.rules.sources=.ifc"}`. By the time an inner host +module is appended, `moduleFlags` already contains `/reference` -- put there by +the bundled `mcpp` module at `hostprogram.cppm:690`. The first token is found, +**skipped**, and only the pair's second half is appended: + + ... /reference mcpp= huxerui.rules.sources= ... + +`cl.exe` reads the orphan as a source file name, which is C1083 verbatim. Clang +is immune by construction: its form is one word and never equals an existing +element. The report's earlier LNK1104 at 2026年9月8日.1 is the same orphan reaching +the link line instead. + +### What answers it + +Append verbatim. The comment states the whole reason the filter exists -- +"repeating it is harmless but noisy" -- so it buys argv tidiness and pays with +a broken command line. If the noise is worth removing, de-duplicate the flag +list **as a sequence** (skip only when `hm->useFlags` already appears in order), +which is correct for all three shapes because it never splits a pair. + +Criterion: a package with a host module that itself declares a host module, +built under `windows = "msvc@system"`, produces an argv in which **every** +`=` token is immediately preceded by `/reference`. Stated as a +count rather than a search, because the defect is a missing occurrence and a +grep for `/reference` finds the one that is there. + +This is a unit-testable statement about flag assembly and should be one: the +end-to-end path needs a Windows runner with MSVC, and the property does not. + +## 3. #603 -- the probe cannot be called as it stands + +The report's diagnosis is right: `src/toolchain/clang.cppm:153-166` hardcodes +`importStdMinLevel = 23` for the MSVC-STL fallback while `msvc.cppm:1040-1042` +probes. Its suggested fix needs one correction. + +`std_module_min_level(const Toolchain& tc)` (`msvc.cppm:934`) reads +**`tc.version`** and compares it against 19.38 -- the cl banner threshold for +microsoft/STL#3977. On the clang path `tc.version` is clang's version, so +calling the existing function there compares a clang version number against an +MSVC threshold: clang 20.x passes it by accident, clang 19.x fails it wrongly. +Both answers would be produced by asking the wrong object, which is the same +error the current hardcode makes, one step less visibly. + +The version that is actually binding is discoverable, and from the file that +was already selected. `find_std_module_source()` (`msvc.cppm:427`) returns + + /Tools/MSVC/14.44.35207/modules/std.ixx + +so the toolset version -- which is the STL's version -- is the parent's parent's +filename. Toolset `14.` and cl banner `19.` share `N` by MSVC convention, +so the existing `>= 38` predicate transfers unchanged. + +### What answers it + +A function that takes the std module source path and returns the level: + + int std_module_min_level_for_stl(const std::filesystem::path& stdIxx); + +with 23 when the path yields no parseable `14.`. Both paths call it, and +the clang path stops being a special case. Preferring the *selected* `std.ixx` +over a fresh `find_msvc_tools_dir()` matters on a machine with two +installations: the answer must describe the STL that will be compiled, not +whichever one the search finds first. + +Criterion: two unit tests over the path shape -- `14.44.35207` answers 20, +`14.37.x` answers 23 -- plus one that a path with no toolset component answers +23. The existing `msvc.cppm` behaviour is unchanged for a real cl, which is +worth asserting too, since that path currently passes for a reason +(`tc.version` is genuinely cl's there) that this change must not disturb. + +## 4. #599 -- a check that has never run in CI + +The report finds two defects. There are three, and the third subsumes the +concern the report raises about the second. + +**(a) The hub path is stale, and the report's replacement is also wrong.** +`bench/matrix.json:136` names `modules/platform/src/platform.cppm` for the +pinned `mcpp-2026年8月11日.3`. The report says the file "lives at +`src/platform.cppm` there". Measured: it is at +`bench/projects/mcpp/mcpp-2026年8月11日.3/src/platform/platform.cppm`. Worth stating +because it is the same error one layer up -- a path written from memory of a +tree rather than read from it. + +**(b) The `uninit` branch prints a note.** `tests/e2e/233_bench_matrix.sh:340-346` +prints `NOTE: hub/body existence NOT checked for ...` and does not fail. A note +is invisible in a green job. + +**(c) There is no bench workflow.** `.github/workflows/` contains sixteen +files and no `bench.yml`; `grep -rn submodules .github/workflows/` matches only +three `--recurse-submodules` clones of *other* repositories. So the test's own +justification -- + +> The bench workflow checks submodules out and runs this test, so the +> assertion does execute on every change to the suite. + +-- is false. The hub/body existence check has run in **zero** CI jobs since it +was written. It fires only on a developer machine that has run +`git submodule update --init`, which is where the report found it. + +### What answers it + +Three parts, and (c) first, because fixing (a) without it fixes one string and +leaves the mechanism that let it rot. + +1. Make the check run where the submodules are. Either restore a bench + workflow that checks them out, or -- cheaper and enough for this + assertion -- add `submodules: true` to the checkout of whichever e2e shard + runs `233`, since the check needs the trees and not the toolchains. The + trees are pins; the cost is a shallow fetch. +2. Make the `uninit` branch fail when it is reached in CI and note when it is + reached locally. The distinction is `CI=true`, which every runner sets. A + developer without submodules must not be blocked; a runner without them is + a mis-configured job, and that is the thing to report. +3. Resolve the hub against the tree rather than against one string. The pinned + tree is historical by construction, so a single path is wrong for it the + moment the layout moves -- which is what happened. A per-project hub keyed + by pin is the minimal fix; resolving by basename within the tree is the one + that survives the next move, at the cost of ambiguity when two files share a + name. Recommend the per-project hub, and record in `matrix.json` that the + path belongs to the pin and not to the repository. + +Criterion: with the submodules absent, `233` fails when `CI=true` and passes +with a note otherwise; with them present, it passes -- and it fails if +`matrix.json`'s hub is edited to any path that tree does not contain. The last +clause is the one that says the check measures something. + +## 5. #564 -- two dead keys that want opposite answers + +Both claims verified. `defaultJobs` and `defaultBackend` each have exactly two +mentions in the tree -- a declaration (`src/config.cppm:96-97`) and a parse +(`:517-518`) -- and no reader. The generated template plants both +(`src/config.cppm:356-358`). + +The report offers one resolution for the pair ("wire it, or drop it"). They +deserve different ones. + +**`default_jobs` should be wired.** It names a property of the machine, and no +other key can hold it. `MCPP_JOBS` must be repeated on every invocation. +`[build] jobs` is per-package, and `[workspace.build]` refuses it +(`modules/manifest/src/toml.cppm:2652-2657`) -- correctly, because a workspace +is not a machine -- so a seven-member workspace would carry the number seven +times and commit a machine fact to the repository. The arithmetic in +`policy.cppm:131-136` is the argument: at 0.5-1.0 GB per module compile, ninja's +own default of 10 on an 8-core, 15 GiB machine swaps. + +Wire it as a **parameter** to `resolve_jobs`, between the manifest and the +backend default: + + MCPP_JOBS> [build] jobs> global default_jobs> 0 (say nothing) + +A parameter rather than an import, because `resolve_jobs` deliberately depends +on nothing but the manifest (`policy.cppm:138-139`). + +The second caller is the part to decide rather than inherit. +`src/build/execute.cppm:2179` uses `resolve_jobs` for test-runner concurrency, +where 0 falls back to `hardware_concurrency()` rather than to a backend +default. Someone who sets a machine-wide number almost certainly means it there +too -- a test runner at 10 concurrent processes has the same memory shape as a +compile at 10 -- so it should apply, and be documented as applying. What must +not happen is for it to apply silently: this is a second behaviour under one +key, and `docs/04-mcpp-toml.md` has to say so. + +**`default_backend` should be removed.** `BackendKind` has `Ninja` and `Native` +(`src/build/backend.cppm:10`) and `src/build/` contains one backend +implementation. The key promises a choice that does not exist, and its default +value `"ninja"` makes it read as implemented. Removing it from the template and +the parser is the honest state; when a second backend ships, the key comes back +with a reader. + +Criterion: an e2e that writes `default_jobs = 3` into a scratch `MCPP_HOME`, +builds, and reads `-j3` off the ninja argv -- and the same fixture with +`MCPP_JOBS=2` set, asserting `-j2`, so the precedence is measured and not just +the plumbing. Four e2e fixtures and one CI action carry a copy of the generated +`config.toml` and each needs the `default_backend` line dropped; a grep for the +key must return zero outside the changelog. + +## 6. #597 -- most of it landed, and the rest is smaller than the report thinks + +The report's four items, against the current branch: + +**(1) triple parsing -- done.** `wasm32-emscripten` is a row in `kKnownTargets` +(`modules/toolchain-model/src/triple.cppm:589`), `parse()` accepts the +`emscripten` OS segment (`:951`), and `llvm_triple()` emits +`wasm32-unknown-emscripten` (`:172`). + +**(2) the object format -- done, and it is why the axis exists.** +`ObjectFormat::Wasm` (`:58`, `:192`) was added as a third value across the +engine precisely because a wasm target has no row in a two-valued +ELF/Mach-O/PE mapping. `family()` answers `unix` for it (`:253`), because +Emscripten supplies a POSIX emulation. + +**(3) toolchain resolution -- not a new compiler family.** The report reads +`emcc`/`em++` as a fourth driver alongside `llvm`/`gcc`/`msvc`. It is not: +`em++` *is* clang, with `--target=wasm32-emscripten` and its own sysroot baked +in, which is the shape the engine already serves. `src/toolchain/hostflags.cppm` +names it: "Every hosted cross this build tool could do was served by a payload +whose driver had exactly one target -- `x86_64-w64-mingw32-g++` needs no +`--target` because it has no choice." `CompilerId` does not need a fourth +value; the payload does, and `xim:emsdk` is published (xim-pkgindex #805). +What remains is the resolver accepting that payload for this triple, which is +the `kKnownTargets` row plus a toolchain layer entry. + +**(4) link semantics -- reuses an existing channel.** The output shape is the +report's one genuine design item and it is smaller than it looks. `em++ -o +app.js` writes `app.js` **and** `app.wasm`. mcpp's link edge already carries +implicit outputs for exactly this -- `ninja_backend.cppm:2162-2174` attaches +Windows import libraries and PDBs to the link edge with ` | ` -- so the `.wasm` +is a sibling on the same edge, not a second target. Execution is the existing +`runner` key (`modules/buildmcpp/src/directives.cppm:280`, generalised beyond +bare metal in #544): `runner = ["node"]`. + +`--preload-file` is the only item with no existing mechanism, and it is not +needed for a first tier: a program that reads no data files at run time is the +common case, and `mcpp pack` is where a data-file policy belongs when it is +needed. + +### What answers it + +The row's tier. `planned` is the honest value today and `verified` is +reachable, because unlike the Android and iOS rows in the same batch, every part +of the loop is on a Linux runner: `xim:emsdk` supplies the driver, `node` +supplies execution, and the artifact can be run and its output compared. The +work is a toolchain layer entry, the implicit `.wasm` output on the link edge, +`runner = ["node"]` in the row's defaults, and a CI lane that builds and runs +a program for the target. + +Recording here rather than deferring: this is the same batch as #605/#607 and +the row already exists, so #597 closes when the row graduates, not with a +separate design. + +## 7. Order, and what depends on what + +Nothing here shares code with anything else here, so the order is by cost of +leaving it in place: + +1. **#606** -- live, deterministic, and downstream. `mcpp-index`'s + `mysql-connector-cpp` fails on every shard that contains it, and the silent + form corrupts module graphs without a diagnostic. It is also the smallest + fix of the six. +2. **#604**, then **#603** -- `msvc@system` is the documented way around #603, + and #604 blocks it, so the pair has an order even though the fixes are + independent. Both are small and both are unit-testable without a Windows + runner. +3. **#599** -- a check that has never run is a check that cannot report the + next stale path. Fixing (c) is what makes (a) stay fixed. +4. **#564** -- a promise the generated file makes and the engine does not keep. +5. **#597** -- graduating a row that already exists, in the batch that added it. + +The first four are one release. #597 belongs to the platform batch. + +## 8. Self-review, and the one plan a measurement changed + +Written after §1-§7 and before any implementation. Four of the six plans +survive unchanged. One is wrong, one has an unstated cost, and the review +found the defect the plan for #606 would have half-fixed. + +### 8.1 #606: the defect is bidirectional, and the other direction is worse + +§1 proposed "a `bool in_block` carried across iterations, with the stripping +done before `strip_line_comment`", and gave as a criterion that +`/* */ import x;` on one line "still records the import". Both are wrong. + +Measured on 2026年9月10日.2, by whether mcpp emits its own +`imported but not provided` warning (which only the scanner can produce, so it +separates "the scanner saw it" from "the compiler saw it"): + +| source | scanner | correct | +|---|---|---| +| `import x;` | sees it | sees it | +| `const char* s = "a /* b";` then `import x;` | sees it | sees it | +| `/* */ import x;` | **misses it** | sees it | +| `// R"(` then `import x;` | **misses it** | sees it | +| `/*`, `R"(`, `*/` then `import x;` | **misses it** | sees it | +| `/*`, `export module y;`, `*/` | records a phantom producer | ignores it | +| `/*`, `module (exe)`, `*/` | refuses the build | ignores it | + +So `/* */ import x;` is not a behaviour to preserve -- it is a fourth wrong +answer. And two of the wrong answers run in the **opposite** direction to the +reported one: a `//`-commented or block-commented raw-string opener puts +`strip_raw_strings` into raw mode, which blanks every following line until a +`)"` that never comes, and real declarations after it are invisible to the +scanner while remaining visible to the compiler. + +A missed `import` is worse in kind than a refused build. It is a **missing +dependency edge**: the compile is not ordered after the BMI it needs, so the +failure is a build-order race that appears under parallelism as +`failed to read compiled module` and disappears on a retry. #606's reported +form is at least deterministic. + +The two directions have one cause. The scanner has three lexical states -- +code, block comment, raw string -- which are mutually exclusive and decided by +whichever opener comes first. It implements one and a half: raw strings fully, +line comments as an unconditional `find("//")`, block comments not at all, and +the three passes run in a fixed order that cannot express "whichever came +first". Fixing block comments alone, in either order relative to the existing +passes, produces one of the two wrong directions: + +* strip comments first, and `R"( /* )"` opens a comment inside a string; +* strip raw strings first, and `// R"(` opens a string inside a comment -- + which is the defect measured above. + +### 8.2 The revised plan for #606 + +One pass over the line with the three states, replacing `strip_raw_strings` and +`strip_line_comment` at the call site. It blanks non-code and preserves +offsets, so the reported column stays correct. State carried across lines is +what it already is (`in_raw`, `raw_close`) plus `in_block`. + +Not a lexer: character and string literals need no tokenising, because the only +question asked of the result is whether the trimmed line *starts with* a +keyword, and an ordinary `"..."` cannot begin a line with one. The one thing +the pass must respect about them is `"a /* b"` -- a `/*` inside an ordinary +string must not open a comment -- which is one state, not a literal parser. + +Criteria, one per row of the table above, with the last two being the pair that +separates a fix from a mute: + +* the phantom-producer case generates a graph with no `gcm.cache/y.gcm` output; +* the four-line file from the report builds; +* `/* */ import x;` records the import -- a *new* property, and the one that a + cheap "skip any line starting with `/*`" would fail; +* `"a /* b"` then `import x;` still records the import -- currently correct by + luck, and the property that stops the fix from treating every `/*` as an + opener. + +### 8.3 #603: one function, and the two answers must be measured to agree + +§3 left open whether the MSVC path keeps the cl banner. It must not: two +readers of one question is what this codebase treats as the defect, and the +`std.ixx` path is the better input on both paths, because it describes the STL +that will actually be compiled rather than the one a fresh search finds first. +The unit test therefore asserts that for a well-formed VC layout the path +answer equals what the banner answer would have been -- otherwise the change +is a silent behaviour change on the one path that was verified. + +### 8.4 #599: the cost of running the check is not stated + +§4 proposes `submodules: true` on the checkout of whichever shard runs `233`. +The bench workloads are pinned full source trees of mcpp and xlings, so this is +not free, and the plan does not say what it costs. Measure before choosing; +if it is large, the cheaper shape is a job that checks out **only** +`bench/projects` and runs `233` alone, since the check needs trees and no +toolchain at all. + +### 8.5 #604 and #564 stand, with one narrowing each + +#604: append verbatim, and de-duplicate nothing. The alternatives considered -- +de-duplicate by logical module name, or by contiguous subsequence -- are both +correct and both add a rule to keep an argv tidy. The rule being removed was +wrong; replacing it with a better rule for the same cosmetic purpose is the +kind of trade this codebase records as a mistake. The comment says the repeat +is harmless; the fix should rely on that sentence rather than work around it. + +#564: the e2e must assert the *precedence*, not the plumbing. A fixture that +only sets `default_jobs` and reads `-j3` would pass if the global value were +wired in above `MCPP_JOBS` instead of below it. Two invocations of one +fixture, with and without `MCPP_JOBS`, is the smallest thing that distinguishes +them. + +### 8.6 #597 stands, and route A now makes a second row measurable + +Unchanged. Noted here because the platform record's Android rows were resolved +by the same kind of measurement in the same session: `qemu-aarch64 -L ` executes the **default, dynamic** +configuration for `aarch64-linux-android`, with `libc++_shared.so` supplied +from the NDK's own directory outside the `-L` prefix. The emulator route is +refuted for every build the vendor manifest currently serves, measured across +all four Linux host entries rather than the pinned one. So both remaining +platform rows are executable on an x86_64 Linux runner with no device and no +virtualization, which is what a CI lane needs. + +## 9. Ecosystem sign-off + +Written after the work landed, across four repositories, against what shipped +rather than against the plan. The question this section answers is not "is each +change correct" -- section 8 and the tests answer that -- but "does the +ecosystem hold together with these changes in it". + +### 9.1 The one thing that went wrong twice, in two repositories + +A check that asks a **proxy** question refuses correct output, and it did so +twice in one day in two repositories: + +* `mcpp-plugins` CI refused a correct MSI. WiX 6 ran with no warnings and + produced 32768 bytes for a 114688-byte program; the floor was + `size> exesize / 2`, and its own comment admitted the ratio was invented. + 32768 is what a stripped hello-world looks like after a cabinet has had it. +* The same repository had already done this with a 16 KB AppImage bound + refusing a correct 14999-byte bundle. + +Both are now direct questions. The AppImage is **run** and its output asserted; +the MSI is **installed** (`msiexec /a`) and the extracted program compared byte +for byte. The pattern to carry forward: when a check reasons "X should be +roughly as large as Y", the artifact can almost always be opened instead. + +### 9.2 The one thing that went wrong twice in the same change + +`#599` was fixed by reasoning about which shard runs `233`, and the answer was +"whichever one the round-robin puts it in" -- so both Linux shards got the +submodules. `233` runs in every job that invokes the whole suite, of which +there are **three**. The macOS lane caught it, which is the same shape as the +defect being fixed: a rule reasoned about against one object and applied to +all of them. + +The correction was to **enumerate**: the three unfiltered +`bash tests/e2e/run_all.sh` jobs are named, and the two jobs that invoke the +suite with a filter or name tests directly are named as not needing it. An +enumeration can be re-checked; a piece of reasoning about sharding cannot. + +### 9.3 What the ecosystem rule turned out to cost, and what it did not + +The rule is the user's: every tool and every library comes from the ecosystem, +and anything missing is added until the loop closes. Applied to the Android +and Web payloads it cost **one new payload** and otherwise only declarations: + +| escape | closed by | new package | +|---|---|---| +| host `debugfs` | `xim:e2fsprogs` | no -- already in the index | +| host `libX11` chain (6 libraries) | declared `deps` | no -- all six already there | +| host `python3` for `em++` | `xim:python@>=3.12` | no, but **aarch64 payload added** | +| `/dev/kvm` | nothing | it is a kernel device | + +The interesting entry is the third. `xim:python` was x86_64-only, and that was +the *stated reason* `xim:emsdk` could not declare an interpreter -- an argument +that was true when written and was an argument for adding the missing payload +rather than for depending on the host. Adding it closed the loop for both +arches. The general form: a dependency declined because the ecosystem cannot +serve it is a request for a package, not a licence to use the host. + +`/dev/kvm` is the boundary the rule has, and stating where a rule stops is part +of stating the rule. A test now asserts it is the **only** remaining warning in +that recipe, so a second one cannot appear quietly. + +### 9.4 Where the ecosystem rule is overruled, and by what + +By the licence, and this is the second time the same framework decided a +packaging question. `xim:iphoneos-sdk` carries no CN mirror because a `CN` +entry would mean xlings-res holds a copy of Apple's SDK. The four Android +packages reach the same conclusion from the same field -- all four declare +`licenses = {"Android Software Development Kit License Agreement"}` -- and keep +one upstream URL each. `xim:emsdk` (MIT / NCSA) and `xim:python` (PSF) are +mirrored because their licences permit it. + +So the rule composes as: **the ecosystem supplies what it may, and the licence +says what it may.** A recipe that declines a mirror should say which of the two +reasons applies, because a reader who cannot tell "not permitted" from "not +done yet" will eventually do the wrong one. + +The cost of getting this order wrong is asymmetric and worth recording: a +mirror that should not exist cannot be withdrawn. Three objects were uploaded +to GitCode before the licence was checked, GitCode assets cannot be deleted, +and the only available remedy is that no recipe references them. The check is +cheap and comes first. + +### 9.5 The cross-repository order, re-derived from what happened + + mcpp engine ──► 2026年9月11日.2 the only thing on the critical path + │ + ├──► xim payloads independent; merged first + │ + └──► mcpp-plugins pins a RELEASE, so it cannot precede one + │ + └──► mcpp-index needs the plugins tag's sha256 + +This was already written in the distribution record's section 11.10, and the +release cycle confirmed it in the sharpest possible way: `mcpp-plugins` #16 has +Linux and Windows green and macOS red on `error: cannot package the Mach-O +program`, which is precisely the defect 2026年9月11日.2 fixes. The dependency is +not a convention -- the red lane *is* the dependency. + +### 9.6 What is still open, stated rather than implied + +* **The four target rows.** `wasm32-emscripten`, `aarch64-ios`, + `aarch64-linux-android` and `x86_64-linux-android` remain `planned`. Both + execution routes are now measured -- `em++` compiles and links `import std` + and `node` runs the result; `qemu-aarch64 -L ` + executes the default dynamic Android configuration -- and both payloads are + published. What remains is engine work of the same size as the distribution + batch: toolchain resolution for two drivers whose target is fixed by their + payload, an implicit `.wasm` output on the link edge, `runner` defaults per + row, 48 matrix cells, and a CI lane per row. That is the next PR, not a + loose end in this one. +* **`xim:wix`** is a legitimate gap with a known shape: MS-RL, a NuGet flat + container, needing `xim:dotnet`. Version 6 and not 7, because 7 refuses to + run without an out-of-band licence acceptance -- a package pinning it would + install a tool that cannot work. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 7fce20b7d..35999c106 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026年09月07日-....md # when status is superseded --- ``` -274 records. +277 records. ## By subject @@ -40,10 +40,25 @@ Records that declare one. Everything else is listed by date below. - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026年09月09日-two-answers-and-two-silences.md) — active +### plugins + +- [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026年09月11日-distribution-plugins-and-platform-decomposition.md) — active + +### targets + +- [Where a platform's knowledge belongs: iOS, Android and Web across the engine, the index and the plugins](2026年09月11日-platform-targets-design-review.md) — active + +### triage + +- [Six open issues: what each one actually is, and what would answer it](2026年09月11日-six-open-issues-analysis.md) — active + ## By date ### 2026-09 +- [Six open issues: what each one actually is, and what would answer it](2026年09月11日-six-open-issues-analysis.md) — active +- [Where a platform's knowledge belongs: iOS, Android and Web across the engine, the index and the plugins](2026年09月11日-platform-targets-design-review.md) — active +- [The category the plugin taxonomy does not name, and what a platform actually decomposes into](2026年09月11日-distribution-plugins-and-platform-decomposition.md) — active - [Two answers and two silences: the scanner's second grammar, and the manifest keys nothing reads](2026年09月09日-two-answers-and-two-silences.md) — active - [A dlopen surface no closure walks, and a process with two unwinders](2026年09月09日-dlopen-surface-and-two-unwinders.md) — landed - [The documentation as a book: a chapter-by-chapter design](2026年09月08日-the-documentation-as-a-book.md) — active diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index c1e67b14a..a8871df73 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -107,7 +107,6 @@ runs: [build] default_jobs = 0 - default_backend = "ninja" EOF cat "$HOME/.mcpp/config.toml" diff --git a/.github/tools/build_examples.sh b/.github/tools/build_examples.sh index 1f4a3b442..de37d3ee9 100755 --- a/.github/tools/build_examples.sh +++ b/.github/tools/build_examples.sh @@ -57,6 +57,19 @@ BUILD=( # three-file package. examples/12-a-new-device-language/toyc examples/12-a-new-device-language/app + # One source, three platforms. BUILT here rather than skipped, because its + # HOST build needs no payload at all -- the `[target.*-linux-android]` + # sections are inert unless that target is selected, which is itself worth + # one build: a manifest that names a target the runner has no payload for + # must still parse and build for the host. + # + # The cross legs are not built here. `wasm32-emscripten` and the two + # Android rows would pull `xim:emsdk` and `xim:android-ndk` -- about 1.5 GB + # between them -- and the signal already exists elsewhere: + # ci-target-matrix scans every row on four hosts, tests/e2e/641 asserts the + # vocabulary, and the example's README records the measured artifacts and + # the emulator run for the binary it describes. + examples/13-platform-targets ) # `key|reason`. diff --git a/.github/tools/check_target_tiers.py b/.github/tools/check_target_tiers.py new file mode 100755 index 000000000..23e32abc3 --- /dev/null +++ b/.github/tools/check_target_tiers.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 +"""Every documented tier agrees with the target table. + +WHY THIS EXISTS. `kKnownTargets` in modules/toolchain-model/src/triple.cppm is +the single source for a row's tier, and four documents restate it: both +READMEs, both copies of docs/21. When `wasm32-emscripten` became `verified` and +the Android rows gained tiers, docs/21 was updated and the READMEs were not -- +so the front page told a reader that three targets were `planned` while the +engine had built and run two of them. Nothing compared the two, which is the +whole reason it could drift. + +THE DENOMINATOR IS THE ENGINE'S TABLE, not the documents'. A check that walked +the documents would pass on a document that lists nothing; this one fails when +a row the engine has is absent from a table that carries tiers at all, and when +a tier disagrees. +""" +import re +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] +TABLE = ROOT / "modules/toolchain-model/src/triple.cppm" +TIERS = ("verified", "preview", "planned") + +# The engine's answer. +rows = {} +for m in re.finditer(r'^\s*\{\s*"([a-z0-9_.+-]+)",\s*"(verified|preview|planned)"', + TABLE.read_text(), re.M): + rows[m.group(1)] = m.group(2) +if len(rows) < 20: + sys.exit(f"ERROR: only {len(rows)} rows parsed from {TABLE.name}; " + "the pattern no longer matches the table") + +# Documents that carry a tier column at all. A document without one is not in +# scope -- prose that mentions a target is not a claim about its tier. +docs = [ + ROOT / "README.md", + ROOT / "README.zh-CN.md", + ROOT / "docs/21-the-target-triple.md", + ROOT / "docs/zh/21-the-target-triple.md", +] + +fail = False +for doc in docs: + if not doc.exists(): + print(f"ERROR: {doc.relative_to(ROOT)} is missing") + fail = True + continue + seen = {} + for line in doc.read_text().splitlines(): + if not line.startswith("|"): + continue + cells = [c.strip() for c in line.strip().strip("|").split("|")] + tier = next((c for c in cells if c in TIERS), None) + if tier is None: + continue + # Every target named in the row's FIRST cell takes the row's tier. + for name in re.findall(r"`([a-z0-9_.+-]+)`", cells[0]): + if name in rows: + seen[name] = tier + if not seen: + print(f"ERROR: {doc.relative_to(ROOT)} is listed here but names no " + f"target with a tier; either it lost its table or this list is stale") + fail = True + continue + for name, tier in sorted(seen.items()): + if rows[name] != tier: + print(f"ERROR: {doc.relative_to(ROOT)}: {name} documented as " + f"'{tier}', the table says '{rows[name]}'") + fail = True + missing = sorted(set(rows) - set(seen)) + if missing: + print(f"ERROR: {doc.relative_to(ROOT)} carries tiers but omits " + f"{len(missing)} row(s): {', '.join(missing)}") + fail = True + print(f" {doc.relative_to(ROOT)}: {len(seen)} of {len(rows)} rows") + +if fail: + sys.exit(1) +print(f"OK: {len(rows)} target tiers agree across {len(docs)} documents") diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 781921205..d3683070e 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -276,8 +276,27 @@ jobs: - distro: ubuntu-2004 image: ubuntu:20.04 setup: apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install curl bash tar gzip xz-utils git ca-certificates binutils findutils file - - distro: debian-11 - image: debian:11 + # debian-12 AND NOT debian-11, AND THE REASON IS NOT THE FAILURE. + # + # The debian-11 leg started failing on 2026-09-11 with + # + # E: Release file for .../bullseye-security/InRelease is expired + # (invalid since 3d 5h 32min 52s) + # + # and `apt-get update` exits 100. Bullseye is end-of-life and its + # security suite's metadata has expired, which is a property of the + # distribution and not of this workflow -- `-o + # Acquire::Check-Valid-Until=false` would silence it and keep a leg + # that tests against metadata nobody maintains. + # + # What was measured while replacing it: debian 11 and ubuntu 20.04 + # both carry glibc 2.31, so the "older glibc" coverage this leg was + # here for was ALREADY DUPLICATED by the ubuntu-2004 leg above, and + # dropping bullseye loses nothing. Bookworm's 2.36 sits between that + # 2.31 and debian-testing's rolling version, so this leg now covers a + # point the matrix did not have. + - distro: debian-12 + image: debian:12 setup: apt-get update && apt-get -y install curl bash tar gzip xz-utils git ca-certificates binutils findutils file env: # The one derived value (see the header comment): every install job names diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 0c0170885..6e427b99f 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -42,7 +42,22 @@ jobs: # set only in the fresh-install workflows (cold bootstrap, no such asserts). # A specific test that needs verbose passes `--verbose` itself. steps: + # `submodules: recursive` so tests/e2e/233_bench_matrix.sh can check that + # each `hub`/`body` in bench/matrix.json exists in the tree it names. + # Without the trees that check reads "submodule not initialised" and + # reports nothing, which is how a hub path written for the CURRENT + # layout stayed in matrix.json while the workload it named is a + # HISTORICAL mcpp -- three cells reporting `skipped` on every bench run + # and the job still green (#599). + # + # It is not free and it is not expensive: the three pinned workloads are + # 725 + 701 + 806 tracked files, under 10 MB of source in total, and + # nothing here builds them. Both shards carry it because run_all.sh + # slices the file list round-robin, so which shard holds 233 moves when + # a test is added. - uses: actions/checkout@v4 + with: + submodules: recursive # Same cache lineage as ci-linux.yml so this job lands on a warm # toolchain/sandbox instead of re-installing it. diff --git a/.github/workflows/ci-linux.yml b/.github/workflows/ci-linux.yml index 1f5a74c4f..90dd66d3d 100644 --- a/.github/workflows/ci-linux.yml +++ b/.github/workflows/ci-linux.yml @@ -108,6 +108,18 @@ jobs: - name: Check documentation structure run: bash .github/tools/check_docs_structure.sh + # Every documented tier agrees with kKnownTargets. + # + # Four documents restate a row's tier and nothing compared them to + # the table. When wasm32-emscripten became `verified` and the + # Android rows gained tiers, docs/21 was updated and both READMEs + # were not -- so the front page said three targets were `planned` + # while the engine had built and run two of them. On its first run + # this check also found two Cortex-A rows missing from docs/21 + # entirely, which predates that work. + - name: Documented target tiers agree with the table + run: python3 .github/tools/check_target_tiers.py + - uses: ./.github/actions/bootstrap-mcpp - name: Configure mirror + Build mcpp from source (self-host) diff --git a/.github/workflows/ci-macos-e2e.yml b/.github/workflows/ci-macos-e2e.yml index 5d02b76e4..7a47e8508 100644 --- a/.github/workflows/ci-macos-e2e.yml +++ b/.github/workflows/ci-macos-e2e.yml @@ -28,7 +28,14 @@ jobs: # NOTE: no MCPP_VERBOSE — the e2e suite asserts mcpp's default quiet # output (tests 48/53). steps: + # `submodules: recursive` so tests/e2e/233_bench_matrix.sh can check that + # each `hub`/`body` in bench/matrix.json exists in the tree it names -- + # the check reads "submodule not initialised" without them and reports + # nothing, which is how a stale hub path survived (#599). Under 10 MB of + # source across the three pins, and nothing here builds them. - uses: actions/checkout@v4 + with: + submodules: recursive - uses: ./.github/actions/setup-macos-llvm - name: Build mcpp from source (self-host) diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index b339df95e..cdd48e338 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -36,7 +36,14 @@ jobs: # assert mcpp's DEFAULT (quiet) output — e.g. 48_build_error_output and # 53_namespaced_cache_label — which forced verbose would break. steps: + # `submodules: recursive` so tests/e2e/233_bench_matrix.sh can check that + # each `hub`/`body` in bench/matrix.json exists in the tree it names -- + # the check reads "submodule not initialised" without them and reports + # nothing, which is how a stale hub path survived (#599). Under 10 MB of + # source across the three pins, and nothing here builds them. - uses: actions/checkout@v4 + with: + submodules: recursive - uses: ./.github/actions/bootstrap-mcpp - name: Build mcpp from source (self-host) diff --git a/CHANGELOG.md b/CHANGELOG.md index cf282c38f..dcf27e6c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,529 @@ ## [Unreleased] +## [2026.9.11.3] - 2026-09-11 + +### `wasm32-emscripten` 从 `planned` 到 `verified` + +`mcpp run --target wasm32-emscripten` 在一个 `import std` 的源码上打印 `1-2-3`, +而工程侧**一个新词汇都不需要** —— 改一个 flag,一个为 Linux 构建的工程就能为 Web +构建。这是「加一个平台等于一次引擎改动」这个代价的第一次实测:七处引擎改动。 + +七处里每一处都是**前一处的失败**找出来的,而每一处都是一个**对它作者心里那些行 +正确**的谓词: + + 1. `to_xim_package` 里由**目标**决定载荷 —— 否则 `xim:llvm` 会回答一个 wasm 目标 + 2. `XimToolchainPackage::frontendSubdir` —— 否则去 `bin/` 里找 `em++` + 3. `Triple::has_own_sysroot()`,放在**共享生产者** `host_compile_tokens` 里 —— + 第一次尝试只改了 `flags.cppm` 一个调用点,于是普通编译不再注入宿主头文件, + 而 std 模块预编译照旧注入,正是它原本失败的地方。一个决定、一个站点、三个读者。 + 4. 同一个谓词放进 `resolve_link_model` —— **模型**而不是它的两条通道;那里的注释 + 早就记着「只修第一条的人会看到一模一样的报错」 + 5. 同一个发现的第二次:`discover_link_runtime_dirs` 把**编译器自己的**运行期目录 + 放上了**产物的**链接行。这条之前的每一行里,两者是同一个目录。 + 6. `host_can_serve`:第一版无条件返回 true,与它上方那些分支是同一种过宽 —— + **目标矩阵抓到的**:`xim:emsdk` 只发布 linux + 7. `Format::Wasm` 及其机制 —— 而这一处**模块自己预言并推迟过**:「Adding + `Format::Wasm` is deferred to whoever gives this module a mechanism for it」。 + 它缺席期间,每一次 wasm 构建都会警告一个这个目标上不可能存在的 `libc++.so`。 + +wasm 同时加入**能力钉**(capability pin):没有别的东西能发出 WebAssembly,所以 +声明一个 `gcc@16.1.0` 是一个无法被满足的请求,说出来比解析出 gcc 再在它内部失败要好。 + +### 两个 Android 行从 `planned` 到 `preview` + +`mcpp build --target aarch64-linux-android` 产出真实的 Android 产物,工程侧除 +`--target` 之外**一个新词汇都不需要**: + +``` +aarch64-linux-android -> ELF 64-bit LSB pie, ARM aarch64, + interpreter /system/bin/linker64 +x86_64-linux-android -> ELF 64-bit LSB pie, x86-64, 同一个 interpreter +``` + +两行**共用一个钉** `android-ndk@30.0.16248370`。这不是省事,而是这条路径的全部 +性质所系:NDK 不命名架构,`--target` 才命名 —— 于是每一处「谁说出目标」的缺口都 +会在这里现形,而在 wasm 上都不会,因为 `em++` 只有一个目标。 + +**一个钉,两个层级**,而层级的差别是**执行**,不是对构建的信心。 + +`x86_64-linux-android` 是 `verified`:产物在平台自己的模拟器上跑起来了。 +2026年09月11日,linux-x86_64,API 24 的 x86_64 系统镜像 + KVM: + +``` +adb push /data/local/tmp/ +adb shell ./andtest -> 1-2-3 exit 0 +``` + +`aarch64-linux-android` 是 `preview`:构建方式完全相同,而从一台 x86_64 宿主没有 +执行路径。记下来是为了下一次尝试不重复:Google 的模拟器直接拒绝异构 guest —— +`QEMU2 emulator does not support arm64 CPU architecture` —— 所以 arm64 镜像需要一台 +arm64 宿主。文档给出的退路是 qemu-user 配系统镜像自带的 bionic,而准备它要用 +`debugfs` 从一个 ext4 分区镜像里取四个文件,而 `debugfs` 恰好是 +`xim:e2fsprogs@1.47.3` 里唯一一个构建坏了的程序(任何打开文件系统的命令都 SIGFPE, +而同一份构建里的 dumpe2fs/e2fsck/tune2fs 都正常)。那是一个生态缺陷,在索引侧有 +自己的记录,不是引擎的缺口 —— 它被修好时这一行就变成 `verified`,而这里什么都不用动。 + +还有一条链接器告警值得记下来,因为用户会看到它而它**不是**缺陷: +`unsupported flags DT_FLAGS_1=0x8000001`。API 24 的 bionic 加载器不认识 lld 设置的 +`DF_1_PIE` 位,于是告警一句,然后照常把程序加载起来。 + +五处引擎缺口,每一处都是前一处的失败找出来的,而每一处都**只在多目标载荷上现形**: + + 1. **std 模块的预编译从来拿不到 `--target`。** `stdModuleTargetFlags` 只到达 + codegen 那一条命令,依据是「第一步要头文件、第二步要机器」。第一步两者都要: + 一个不说目标的 `--precompile` 会把标准库自己的 `#include <__config>` 解析到 + **正在构建的那台机器**上。而这个文件里早就记着同一句报错 —— 2026-08 一台 + Windows 宿主上的 `'__config' file not found` —— 同一个成因,另一条路径到达。 + 现在预编译从两个来源里**有机器的那一个**取:`stdModuleFlags` 非空时它是 + `stdModuleTargetFlags` 的超集,所以取它就不会让 `--target` 上两次命令行。 + 2. **`-D__BIONIC_CTYPE_INLINE=`。** bionic 把 `isalnum` 一族声明为 `static + inline`,而 libc++ 的模块面用 `using std::isalnum` 导出它们 —— using 声明不能 + 导出内部链接的名字,于是预编译一次性在 14 个名字上失败。范围限定在 std 模块 + 上:被满足的规则是「从模块里导出」,而一个直接 `#include ` 的翻译 + 单元有权拿到 bionic 的 inline 定义。`xim:android-ndk` 自己的安装期自检从另一个 + 方向得到同一个结论。 + 3. **「自带 sysroot 就什么都不告诉它」比实际强了一个 token。** 那个提前返回 + 站在 `host_compile_tokens` 里「THE TRIPLE, SAID OUT LOUD」那一段**之前**,而 + 那一段讲的是相反的规则、理由相同:一个普通 clang 不被告知就为它自己所在的 + 机器发码。两句话各自对它自己的对象是对的 —— **体系**是载荷的、不可重建; + **是哪个目标**仍是 mcpp 要说的。报出这件事的是模块加载器而不是任何一次编译: + 「AST file 'std.pcm' was compiled for the target + 'aarch64-unknown-linux-android21' but the current translation unit is being + compiled for target 'x86_64-unknown-linux-gnu'」,后面跟着八条级联的 + 「use of undeclared identifier 'std'」,而读者先看到的是后者。 + 4. **链接行同样没有目标。** 两条链接分支都被跳过,而它们携带的东西被跳过是对的: + C 库、C++ 运行期、crt 对象和加载器全在 SDK 里,驱动自己会找。它**不能**猜的 + 是找哪一个。落空的结果是目标的对象与宿主的启动文件链在一起:六个宿主对象, + 每一个都由一个以为自己在为本机构建的驱动解析出来。新增的第三条分支只放 + `crossTarget`,别的什么都不放。 + 5. **`discover_link_runtime_dirs` 的闸在错误的时刻求值。** 那个闸本身是对的 —— + 它拒绝为自带 sysroot 的目标报告这些目录 —— 而它跑在**探测期**,比目标被赋值 + 更早,那时 `targetTriple` 还是宿主的。产物是揭发它的东西:一条 Android 链接行 + 上带着 `-L /.../prebuilt/linux-x86_64/lib/x86_64-unknown-linux-gnu`, + 最后一段是本机的 triple,由 `root / "lib" / targetTriple` 产生 —— 那个字符串 + 写着被问出来的那个问题。 + +还有两处**产物检查**把一个正确的产物判成了缺陷,而两处的形状相同:一条对**面向 +宿主**的产物为真的规则,施加在一个交叉产物上。 + + - **hermetic 链接检查**把 `/system/bin/linker64` 算成「sandbox 之外」。它是这个 + 函数检查的所有路径里唯一**不在本机解析**的那一个:它被记进产物,由**设备**在 + 加载时读取,而 Android 的这条路径由 ABI 规定,不可能在任何载荷里面。消息对 + 它看到的东西是准确的、对它的含义是错的 —— 这是更难的那一种,它点名了一条确实 + 在 sandbox 之外的真实路径,并邀请读者去重装一个与此无关的 glibc 载荷。 + - **runtime 闭包校验的 rule B**拿产物的 `PT_INTERP` 与**宿主的** `RuntimeBinding + glibc@2.44` 相比,报「one process cannot mix runtime payloads」—— 一句关于一个 + 永不会存在的进程的真话,接着建议一个帮不上忙的 SubOS。上方那个 linux/glibc + 闸拦不住它:Android 的 triple 的 `os` **就是** `linux`,这是刻意的。 + +Android 同时加入**能力钉**,而它的理由与另外三个都不同。那三个被拒绝是因为工具链 +发不出那个**格式**;一个普通 clang 发 aarch64 ELF 完全没问题。它拿不出来的是 +**体系** —— bionic 的头文件、按 API level 的存根、加载器路径都在 NDK 里,没有任何 +包能把它们加到另一个编译器上。 + +### `host_can_serve` 不再把「Linux」编译进引擎 + +这一行此前对自带 sysroot 的目标返回 `mcpp::platform::is_linux`,而它自己的注释就 +写着这个判断的失效条件:「When a darwin or windows NDK lands in the index — +upstream publishes both — this is the one line that changes.」它落地了,所以这就是 +那一行。 + +同一个 goal 的两半随即互相矛盾:索引把载荷发布到三个宿主,而引擎在其中两个上 +**把这一行从 `toolchain list` 里删掉**。症状不是一个错答案而是一个**缺席的**答案 —— +macOS 上 `mcpp build --target wasm32-emscripten` 报了一个这张表认识的目标「未知」, +正是 `planned` 这个层级存在的目的所要避免的事。 + +两半都不是它被发现的地方。`scan (macos-arm64)` 与 `scan (windows-x86_64)` 在**格数** +上失败 —— 测到 24 格、表里声明 25 格 —— 而那唯一缺的一格点名了这一行。一个按宿主 +跑、拿一张入库的表作判据的 job,是这个仓库里唯一能看见**一行消失**的东西,因为其他 +每一处检查问的都是它已经拿到的那一行。 + +那条闸的单测同样是**在一台宿主上由算术为真**:它写的是 +`EXPECT_EQ(host_can_serve(*wasm), mcpp::platform::is_linux)`,而在 Linux 上 +`is_linux` **就是** `true`。一个期望值等于它所运行的宿主的判据,报不出另两个宿主上 +的变化。现在它无条件断言,并且补上了「这个谓词仍然说得出『不』」那一半。 + +### 模拟器是一行,不是一个 runner + +一次 iOS 模拟器构建是**另一个目标**:它自己的 SDK(`iPhoneSimulator.sdk`)、自己的 +对象,取 `-mios-simulator-version-min` 而真机取 `-miphoneos-version-min`。此前它 +无法被拼写,而设备那一行的注释把反对意见写反了 —— 反对的是**没有**一个单独的行, +而不是有。 + +`env = "sim"` 给出 `aarch64-ios-sim` 与 `x86_64-ios-sim`,也就是 Rust 的 +`aarch64-apple-ios-sim` 减去这张表本来就省略的 vendor 段。Apple 自己的 +`-simulator` 拼法也解析到同一行 —— clang 打印的 effective triple 带的是那一种,而 +一个把它粘回来的读者不该被告知 mcpp 从没听说过它。 + +两个架构都有,理由与 Android 那一对相同:模拟器跑**宿主的**架构,所以一行会描述 +一半机器跑不了的模拟器。 + +两行都是 `planned`,而阻塞项与设备行是同一个**许可**问题,不是一个载荷问题:NDK 是 +Apache-2.0、Emscripten 是 MIT,而 iPhoneOS 与 iPhoneSimulator 的 SDK 在 Xcode 里, +两者都不可再分发。它们今天买到的是:`mcpp build --target aarch64-ios-sim` 答 +`tier-planned` 并点名那一行,而不是答 `unknown target` —— 后者是假的。 + +### API level 的默认值取自载荷,而「不写」不是一个合法答案 + +`min_platform_version` 此前在项目没有声明 `min_api_level` 时返回空串,注释写的是 +「the NDK's own default, which clang supplies」。那句话从未被验证,而且是错的: + +``` +--target=aarch64-unknown-linux-android (无级别) +sys/cdefs.h:365:2: error: Unversioned target triples are not supported! +``` + +bionic 直接拒绝一个不带版本的 triple,所以级别是**强制**的,一个从没听说过 API +level 的工程也需要一个。于是问题变成这个数字从哪里来。不是一个编译进来的常量: +这个仓库已经记过不止一次,一个写进注释的版本号会变成一个写进诊断的版本号,再变成 +某人 install 命令里的版本号,而 NDK 的下限随 NDK 移动。载荷自己回答 —— +`meta/platforms.json` 是上游自己声明的支持区间,r30 是 `{"min": 21, "max": 37}` —— +于是一个更新的 NDK 靠**被安装**改变这个默认值,而不是靠被编辑进这个文件。 + +读不到时返回 0,由调用方转成一个点名 `min_api_level` 的拒绝。一个**猜**出来的级别 +比这个拒绝更坏:它决定哪些 bionic 符号存在,所以猜会产出一个在这里链接得上、在 +设备上加载不起来的产物。 + +同一句错误假设在 `llvm_triple()` 的注释里还有**第二份**,也一并改掉了。 + +### 一个裸 `aarch64-linux` 永不被补全成 Android + +Android 那两行与 `aarch64-linux-musl` 落在同一个 `arch-os` 前缀上,因为它的内核 +**就是** Linux —— 那正是树里每一处 Linux 形状的答案对它都成立的原因。这不使 bionic +成为一个没有命名 C 库的请求的候选:它有不同的加载器路径、不同的 SDK 和一个 API +level。 + +这件事在两行离开 `planned` 的那一刻变得可达:`aarch64-linux` 于是有了**两个**受支持 +的兄弟行并解析为 ambiguous,而在此之前它补全成 `aarch64-linux-musl`。这个二义性的 +两种结果都是错的 —— 拒绝一个有显然答案的请求,或者用 bionic 回答它。 + +也从 `siblings` 里排除,不只是从 `supported` 里:那个列表是诊断打印的东西,把 +`aarch64-linux-android` 提供给一个输入了 `aarch64-linux` 的人,是在建议他为另一个 +平台构建。一个**写出来的** `aarch64-linux-android` 永远到不了这个循环 —— 显式的 env +在上面就返回了,这条规则是「作者自己的拼写是一个请求而不是一个缺口」。 + +### effective triple 里的 API level 现在能被解析回来 + +`Target aarch64-linux-android -> aarch64-unknown-linux-android21` 是 mcpp **自己 +打印**的一行,而把它粘回去得到的是 `unknown target`:env 段的匹配写的是 +`k == "android"`,而 API level 骑在那一段上。下面 msvc 那条分支为同一个理由早就带着 +同一条注释(`...-windows-msvc19.44.35211`);Android 是同一个形状,被漏掉了。一个前缀 +匹配覆盖全部四种拼法:`android`、`android21`、`androideabi`、`androideabi21`。 + +### 一个按目标付费的机器级扫描,17948ms → 4ms + +`mcpp test` 在这台机器上从约 3 分钟变成投影 33 分钟,而根因不是回归而是**一直 +存在的形状**被一次大载荷安装放大了: + +``` +mcpp build loader-tags 阶段 170ms 一个 21 MB 二进制 +mcpp test loader-tags 阶段 17948ms 108 个二进制、2.4 GB +``` + +而且在测到的每一个目标上都**平的** 17.9 秒 —— 所以代价是**整个产物集合**而不是 +正在构建的那一个。`mcpp test` 每个目标驱动一次后端,于是 110 个目标付了 110 次。 + +`check_dlopen_surface` 会对**每一个已链接产物**做完整 `inspect_elf_runtime` 去收集 +SONAME,而这发生在它发现「surface 是空的」**之前**。它每次写下的记录都说 +`members=0, walked=0`。而那些产物全是**可执行文件**,按构造不可能有 `DT_SONAME`。 + +两处都修了,而值得命名的是形状:**昂贵的工作跑在了那个使它变得不必要的便宜判据 +之前。** 记录照旧发布 —— 一个会消失的字段比一个说明自己为何为空的字段更坏。 + +### `min_api_level`,复用 `macos_deployment_target` 已有的那套机制 + +Android 的 API level 在**交给编译器的** triple 里(实测: +`clang -target aarch64-linux-android21 -print-effective-triple` 答 +`aarch64-unknown-linux-android21`),而它**不该进规范 triple**:mcpp 维护自己的目标 +词汇并映射到编译器目标,而 macOS 早就是这个形状。 + +``` +[target.aarch64-linux-android] +min_api_level = 24 +``` + +规范 triple 保持 `aarch64-linux-android`(它命名输出目录、`cfg(env=)`、ABI tag); +级别由 `llvm_triple()` **已有的那个参数**拼进 effective triple;而它进**指纹** —— +级别决定哪些 bionic 符号可见,所以两个级别是两个 ABI,绝不可共用一个构建目录。 + +字段名取自 Android 自己的词汇:NDK 的 CMake toolchain 把 `ANDROID_PLATFORM` 记载为 +「the minimum API level supported by the application or library」。`ndk_api_version` +被否掉有两条理由:「version」不是 Android 的用词(是 **level**),而 `ndk_` 命名的是 +**工具链**,可一个 NDK 服务一个级别**区间** —— 用 NDK 命名会把这个区分重新弄混。 + +指纹里那个槽因此从 `macosDeploymentTarget` 改名为 `minPlatformVersion`:一个目标 +要么是 Apple 要么是 Android,所以一个槽装不下两者,而两者回答的是同一个问题。改名 +不额外增加任何重建 —— mcpp 版本本来就在这个键里。 + +### 能力钉的理由必须属于它自己那一行 + +`prepare.cppm` 的注释早就把规则写明了:「一句话覆盖两者,就会对其中一个是错的 —— +PE+musl 目标不是裸机,而一个被告知它是裸机的读者会停止阅读。」而我加了第三行却 +没有加第三条理由,于是那句话对新来的那一行成了错的。 + +实测:`--target wasm32-emscripten` 声明 gcc 时**拒绝是对的**,而解释是「No gcc +payload emits a PE with a musl C library」—— 一句关于另一行的、本身为真的话。 + +**而那个闸问错了问题。** 它测 `family != Llvm`,这在「每个能力钉行都钉 llvm」时是 +对的。`wasm32-emscripten` 钉 `emsdk@6.0.9`,而 emsdk **归一到 llvm 族**(因为 +`em++` 就是 clang)—— 所以声明 `llvm@22.1.8` **通过了这个闸、从未被拒绝**,并为一个 +它发不出来的目标解析了通用 llvm 载荷。判据现在是**那一行自己的钉**,也就是这一行 +一直在回答的那个问题。 + +**然后同一件事发生了第二次。** Android 这一行拿到钉、成为能力钉行,而理由链仍是 +三条臂,于是它被解释成了 PE+musl 那一句 —— 与上一段记的一模一样的错答案,经由 +一模一样的路径:第四个情形落进了一个按第三个情形写的 `else`。 + +所以修法不是再加一条臂。最后那条臂现在**点名它自己那一行**(`is_pe() && is_musl()`), +而兜底句是通用的:「This row's toolchain is the only one that can emit the target at +all.」以后新增的能力行拿到的是一句**不够具体**的话,而不是一句**假**话。 + +`tests/e2e/640` 现在钉住九条:四行各自的句子(裸机、PE+musl、wasm、Android)、 +「声明 llvm 也必须被拒绝」这个缺口在 wasm 与 Android 两处、收尾那句要点名本行的钉, +以及**第六条是穷举的** —— 前面每一条都是有人想到了那一行才写下的,而两次缺陷都是 +**没人想到的那一行**掉进了兜底,任何按行写的测试都抓不到。第六条从引擎自己的词汇 +里取出每一个带钉的行,声明一个不是它的钉的工具链,并断言 PE+musl 那句话恰好出现 +在一行上。实测:34 个目标里 17 个是能力钉行,PE+musl 出现 1 次。分母取自 +`toolchain list`,所以明天新增的一行不需要编辑这个文件就已经在里面。 + +### 解析行里说出**是哪个载荷**回答的 + +`emsdk@6.0.9` 归一到 llvm 族 —— `em++` 就是 clang,而第四个 family 值会是对编译器 +的假陈述。代价是那行输出读作 `Resolved llvm@6.0.9`:与真正的 `xim:llvm` 无法区分, +而且不是用户输入的东西。`mcpp toolchain list` 有同样的问题,而矩阵扫描**每族只取 +一个**工具链,所以一台宿主上两个 llvm 族的载荷无法都被枚举。 + +族与载荷是两个问题: + +``` +族 这个编译器说哪一套 flag 词汇? llvm +载荷 哪个归档提供它? emsdk +``` + +`to_xim_package` 早就从目标回答了第二个;新增的字段只是让它**能被说出来**。空值 +表示「该族自己的载荷」,也就是除这两行之外的每一行 —— 所以没有任何既有输出改变。 + +### 四段式拼法被接受 + +`em++ -v` 传给它自己 clang 的是 `-target wasm32-unknown-emscripten`,rustc 的表里 +也是这个拼法。拒绝每个别的工具链都会打印的那个形式是纯 UX 代价。`parse()` 现在接受 +它并规范化,而 `str()` 两边都返回三段式 —— 这才是让输出目录、`cfg()` 和 ABI tag +保持单值的东西。 + +### 那条 EOL 的 distro 腿换掉了 + +`debian-11` 从 2026年09月11日 起以 `E: Release file ... is expired` 失败 —— bullseye +已经 EOL,而它 security suite 的元数据过期了。这是发行版的属性而不是本工作流的, +而 `-o Acquire::Check-Valid-Until=false` 只会让它安静下来并留着一条「对着没人维护的 +元数据测试」的腿。 + +换掉时顺手测了一件事:debian 11 与 ubuntu 20.04 **都是 glibc 2.31**,所以这条腿 +原本要覆盖的「更老的 glibc」**早就被上面那条 ubuntu-2004 腿覆盖了**。bookworm 的 +2.36 落在那个 2.31 与 debian-testing 的滚动版本之间,所以这条腿现在盖住了矩阵原本 +没有的一个点。 + + +## [2026年9月11日.2] - 2026年09月11日 + +### 扫描器读到了注释里面,而且是双向的 + +**报告的那半。** `/*` 单独占一行、下一行是 `module (`,那一行被当成模块声明匹配, +四行普通 C++ 被拒绝(#606)。报告把它二分到「2026年9月7日.1 之后的回归」。实测: +`git log -S` 在 `src/modgraph/scanner.cppm` 上返回**零个**曾添加块注释状态的提交 —— +它从来没有过。2026年9月9日.1(#594)加的畸形名拒绝是**对的**,二分定位到的是缺陷**变响** +的时间。 + +**没被报告、而更坏的那半。** 名字合法时那条拒绝不会触发,结果是:块注释里的 +`export module y;` 让一个普通 `.cpp` 被记录成 `gcm.cache/y.gcm` 的**生产者**,承诺一个 +编译器永不写出的 BMI。真正 `import y;` 的文件于是被告知 `imports must be built +before being imported` —— 一个并不存在的顺序问题,而真正的提供者从未被查找。 + +**以及反方向。** `// R"(` 让 raw string 那一遍进入它出不来的状态,把之后每一行都 +抹空到一个永不出现的 `)"`;`import x;` 对扫描器不可见而对编译器可见。**那是一条 +缺失的依赖边** —— 并行下的构建顺序竞争,重试就好,比报告的那条更坏。`/* */ import +x;` 同样被漏掉,是报告没有提到的第四个错答案。 + +根因是一段被当成已完成的论证,写在源码注释里:「普通 `"..."` 字符串故意原样保留: +匹配器只在 trim 后**以关键字开头**的行上触发,而字符串体只有跨行(即 raw string) +才做得到这件事。」前提成立,枚举少了一个 —— **块注释也能**。 + +修法是**一遍走三个状态**:代码、块注释、raw string 互斥,由先出现的那个开启符决定, +这是任何固定顺序的分遍都表达不了的。判据八条(`tests/e2e/639`),其中两条是 +「原本就对、不能弄坏」的:`/* */ import x;` 必须**看见**那个 import(否决「跳过任何 +以 `/*` 开头的行」这种哑修法),`"a /* b"` 必须**不**开启注释。拿已发布的 +2026年9月10日.2 对照:八条里六条变红。 + +### 一个两 token 的开关丢掉了它的开关 + +`windows = "msvc@system"` 下,一个带 `host-module` 构建依赖的包在编译构建程序时失败: + +``` +c1xx: fatal error C1083: Cannot open source file: + 'huxerui.rules.sources=...\huxerui.rules.sources.ifc' +``` + +`cl.exe` 把模块引用读成了**源文件名**。成因是 host-module 的 flag 收集**按 token** +去重,而它是为唯一一个「单 token 且幂等」的家族写的 —— GCC 的 `-fmodules`: + +| 家族 | useFlags | token | +|---|---|---| +| GCC | `-fmodules` | 1,幂等 | +| Clang | `-fmodule-file==` | 1,唯一 | +| MSVC | `/reference`, `=` | **2,第一个合法重复** | + +内层 host module 追加时,`/reference` 已由内置 `mcpp` 模块放进列表 → 被跳过,只追加 +了对的后半。Clang 按构造免疫(一个词,永不等于已有元素),所以缺陷专属于那唯一一个 +在 Windows 上能在 c++20 达到 `import std;` 的工具链选择(#604)。 + +**逐字追加,不再去重。** 被替换的那句注释说明了这个过滤器的全部价值:「repeating it +is harmless but noisy」—— 它买的是 argv 整洁,付的是坏命令行。按逻辑模块名去重、 +或按连续子序列去重,两者都正确,而两者都是为同一个装饰性目的**新增一条规则**。规则 +被删掉了。 + +另加 `mcpp::toolchain::orphaned_reference`:一个 `=` 前面没有开关时, +在命令跑之前拒绝并说明,而不是从 cl 的 C1083 里去反推。这是纵深防御而不是修复本身。 + +### `import std;` 的档位问的是 STL,不是碰巧到达它的那个编译器 + +clang 在 Windows 上回落到 MSVC STL 的 `std.ixx` 时把 `importStdMinLevel` 硬编码成 +23,于是一个 c++20 工程在 Windows 上被拒绝,而同样的源码在 Linux 的 GCC 16.1、 +Linux 的 llvm 22.1.8 和 macOS 上都能构建(#603)。 + +那段注释把理由说对了 ——「`tc.version` 是 clang 的,所以它回答不了 cl banner 的问 +题」—— 却从中得出了错的结论。**照原样调用已有探针会更坏**:那会拿一个 clang 的版本号 +去和 MSVC 的 19.38 门槛比,clang 20.x 侥幸通过、clang 19.x 错误地答 23,两个答案都 +来自问错对象。 + +真正有约束力的版本在**刚刚选中的那个模块源文件的路径里** —— +`/VC/Tools/MSVC/14.44.35207/modules/std.ixx` —— 而 toolset `14.` 与 cl banner +`19.` 配对,所以现有的 `>= 38` 谓词原样迁移。新增 +`std_module_min_level_for_stl(path)`,**两条路径都调它**;单测断言对一个规整的安装, +两种形态给出相同答案 —— 否则这次改动就是对唯一被验证过的那条路径的静默行为变更。 + +### `[build] default_jobs` 有了读者,`default_backend` 被删掉 + +两个键都被解析、都没有任何消费者(#564)。它们要的是**相反**的答案。 + +`default_jobs` 接上:它是三级优先级里唯一能承载**机器事实**的一级 +(`MCPP_JOBS`> `[build] jobs`> `default_jobs`> 0)。`--jobs` 每次调用都要重说, +`[build] jobs` 是按包的而 `[workspace.build]` 正确地拒绝它,所以别无他处。它以 +**参数**而不是 config import 的形式到达 `resolve_jobs`,因为那个函数刻意只依赖 +manifest 和宿主。它**同时**约束 `mcpp test` 的并发,这一点被明写进文档 —— 一个键 +两种行为必须明说。单测断言的是**顺序**而不是接线:只设全局值再读回来的夹具,在参数 +被接到 `MCPP_JOBS` **之上**时同样会通过。 + +`default_backend` 删掉:`BackendKind` 有两个值而 `src/build/` 只有一个后端实现, +这个键承诺了一个不存在的选择,而默认值 `"ninja"` 让它看起来是实现了的。四个 e2e +夹具和一个 CI action 各自持有一份生成文件的副本,都同步了。 + +### bench 的 hub 检查从写下起在 CI 里跑过零次 + +`matrix.json` 给 mcpp 工作负载写的 hub 是 `modules/platform/src/platform.cppm`,而那个 +工作负载是一个**历史** mcpp,该文件在其中位于 `src/platform/platform.cppm`(#599)。 +三个 cell 的每一个扰动场景都会报 `skipped` 而 job 照常绿。 + +报告发现两个缺陷,实际有三个,而第三个吞掉了前两个:**`.github/workflows/` 里没有 +bench workflow**,也没有任何 job 检出 submodule。所以 233 自己那句理由 ——「bench +workflow 会检出 submodule 并跑这个测试」—— 是假的。 + +三处都修:hub 路径按 pin 重读(并在 `matrix.json` 里记下「hub/body 属于 pin 而不属于 +本仓库」),`uninit` 分支在 `CI=true` 下**失败**而在本地打印提示(两类读者的答案相反), +以及 `ci-linux-e2e.yml` 的两个 shard 都带上 `submodules: recursive` —— 三个 pin 合计 +2232 个 tracked 文件、不到 10 MB 源码,而且这里什么都不构建。 + +### 暂存对被分派的格式是**服务**,不是前置条件 + +`mcpp pack --format ` 在分派之前无条件先暂存一次,而暂存失败就让整条命令失 +败。对 `--format tar` / `--format dir` 这是对的 —— 暂存树**就是**产物。对一个**被 +分派**的格式,它是提供方可能要、也可能不要的一项输入,而把它当成前置条件,会让 +「内建打包被拒绝」的任何目标上,**所有**被分派的格式都变得不可达。 + +实测(macos-15,mcpp 2026年9月11日.1):`mcpp pack --format app` 根本到不了分派 —— +`pack::run` 会直接拒绝一个 Mach-O **程序**,因为内建的闭包走的是 +`LD_TRACE_LOADED_OBJECTS`,那是 glibc 的机制,dyld 不认它、而是**直接把程序跑起来**。 +那条拒绝对内建归档是正确的,却对「一个 `.app` 打包器能不能工作」什么都没说 —— +一个只点名一个程序的打包器根本不需要走闭包。**引擎在回答一个提供方没有被问到的 +问题。** + +所以失败现在是**带着原因继续**而不是被吞掉:原因作为 warning 印出来, +`pack_stage_dir` 保持为空,`${mcpp.stage_dir}` 于是在展开处带着那条原因拒绝。读树的 +提供方拿到精确诊断,不读树的照常走完。没有任何东西被静默降级 —— 变的是**由谁来做 +这个决定**。 + +`BuildOverrides::pack_stage_reason` 是那条原因的通道。没有它,一个明明在打包的构建 +会读到「this build is not packaging」,而那句话会把成员作者引向错误的方向。 + +⚠️ 这条修复是由 CI 在 macOS 上第一次真的跑 `dist-apple` 才暴露出来的。在那之前它 +只有 plan 级断言撑着,而 plan 级断言说的是「闸对了」,对「工具接不接受成员渲染出来 +的东西」一个字都没说。 + + +## [2026年9月11日.1] - 2026年09月11日 + +### `mcpp pack --format ` 分派到包,而引擎里不再需要住进任何一种分发格式 + +`tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 +形状 —— 所以它们是一个 flag 的取值,而不是第二个 flag 的开端。分界是:`mcpp pack` +拥有机制,以及那一种通用格式(一个解开就能跑的归档,它不需要知道任何别人的发布); +其余每一种格式都住在包里。dpkg 的 control 字段、AppImage 的 runtime、WiX 的 schema、 +Apple 的公证,其中任何一个被绑进引擎,都会把一次 mcpp 的发布耦合到一次 mcpp 并不控制 +的发布上。这与本项目早已为语言做过的论证是同一个 —— Slang 被支持,而引擎里没有它的 +名字。 + +三样与格式无关的东西被加进引擎,「与格式无关」正是判断某样东西该不该进引擎的判据: + +- **一棵 `artifact` action 可以消费的暂存树。** `mcpp pack` 一直在算它 —— 依赖闭包, + 过了 strip 策略、调试信息拆分与 `include`/`exclude` —— 然后把它压掉,目录就没了。 + `${mcpp.stage_dir}` 把它暴露出来。它是一个 **bundle** 树(`bin/`、`lib/`、可重定位), + 这正是 AppImage、`.app`、`.msi` 要的形状;要一棵 FHS 树的格式(`.deb`、`.rpm`)自己 + 负责重排,因为一个文件该落在哪个目录是那个格式的知识。 +- **`[package]` 的其余字段进入构建程序。** `MCPP_PKG_VERSION` / `_DESCRIPTION` / + `_LICENSE` / `_AUTHORS` / `_REPO`,以及对应的 `mcpp::package_*()`。每一种安装包格式 + 都要写版本号;在这之前,项目只能把版本号在成员自己的 options 里再写一遍,而那份副本 + 会与 `[package]` 漂移,且没有任何东西能发现。 +- **`--format` 经图解析它的取值。** 包用 `mcpp::provides_pack_format("")` 声明, + `--format ` 在解析后的依赖里找到提供方并把暂存树交给它。未知取值点名**当下确实 + 可用**的那些,而不是一份固定清单,并且这次拒绝发生在任何东西被编译之前。 + +**无条件声明,有条件提交。** 这是整套分派最承重的一条规则,也是最容易被成员作者写错 +的一条 —— 因为写错了对作者自己仍然照常工作:他永远传的是自己那个格式。声明必须不加闸, +否则引擎永远回答不出「这张图提供哪些格式」;提交必须加闸,否则普通 `mcpp build` 会多出 +一条它不该有的边。对一个谁都没为之提交的格式,mcpp 会拒绝并点名,而不是报告一次「什么 +包都没产出」的成功打包。 + +**`mcpp pack --format ` 会 prepare 两次,而两次之间没有任何值被重新推导。** 一条 +`artifact` action 是一条 ninja 边,而暂存树是 mcpp 在链接**之后**产出的,所以这棵树不 +可能成为构建出它自己那一次 pass 的输入。第一趟收集声明并拒绝未知格式;随后是构建与暂存; +第二趟设上 `pack_format` 与 `pack_stage_dir` 并构建提供方提交的那条边。第二趟用的三元组 +与暂存路径,都是第一趟和 `make_plan` 已经回答过的 —— 重新推导一遍会得到那种「在作者所有 +机器上都一致、只在他没有的那台上不一致」的缺陷。 + +**build.ninja 的头行多了第四个字段 `dist=`,而快路径要求它读作 `none`。** 格式故意 +**不进指纹**:进了就要为「把一棵已经构建好的树打成包」付一次全量重编。于是两张图落在同 +一个目录里,而 `target/
//build.ninja` 是被两条快路径回放的共享可变状态 —— +这正是这一行上另外两个字段(`graph=`、`accel=`)已经各自记过一次的那种失败。判据放在 +单元测试里而不是端到端:实测(2026年09月11日)即使忽略这个字段,pack 之后的普通构建也会因为 +更早的一个新鲜度条件而重新生成图,所以端到端断言无论字段是否生效都会通过,连字段被删掉 +都照样通过。 + +**`${mcpp.stage_dir}` 在两种位置上是拒绝而不是空展开:** 本次构建不在打包时,以及 +role 不是 `artifact` 时。一个空路径仍然是命令接受的 token,而工具随后读到的是构建目录 +根 —— 那个目录存在,所以这个错误会产出一个看起来合理的产物而不是一条诊断。实测过的原型 +是那个「有效的、空的、52 KB 的安装包,并且没有任何诊断」。 + +写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对 `<暂存树>.stage-manifest` 的依赖 +(一个兄弟文件,永不是成员,所以它不会跑进任何人的安装包里)。依赖由引擎添加,因为「用 +了」本身就意味着「依赖」:没有它,这条边只在链接产物变化时才变脏,而一个闭包多出了某个 +依赖的共享库、同时程序自己的字节没变的情况,会把上一次的可分发物原地留下并报告为最新。 + +构建程序协议升到 v9(`mcpp:pack-format=`)。这条指令带非空 `tag`,因此会随构建程序的 +缓存记录一起被回放 —— 读它的那一趟是 `mcpp pack`,而那从来不是一个项目的第一次构建。 + +文档:`docs/10`(`--format` 那一个轴)、`docs/30`(三类成员的分类表、新占位符与新访问器)、 +`docs/31`(分发成员的六条约束),中英双份。 + + ## [2026年9月10日.2] - 2026年09月10日 ### dlopen 面检查:不适用的那一趟也会发布记录,并且不会盖掉已经量出来的答案 diff --git a/README.md b/README.md index 578ec150e..8c2720879 100644 --- a/README.md +++ b/README.md @@ -419,6 +419,10 @@ list` reports for this machine): | `aarch64-none-elf` · `x86_64-none-elf` | llvm 22 — bare metal, no C library by default 2 | preview | | `thumbv7em-none-eabi` · `thumbv8m.base-none-eabi` · `thumbv8m.main-none-eabihf` | llvm 22 — Cortex-M4/M7 soft float, M23, M33F/M55F 2 | preview | | `riscv64-linux-musl` · `aarch64-linux-gnu` · `x86_64-macos` | — | planned | +| `wasm32-emscripten` | `emsdk@6.0.9` — Emscripten ships its own sysroot and its own libc++ module surface; `mcpp run` executes the module with `node` | verified | +| `x86_64-linux-android` | `android-ndk@30.0.16248370` — bionic from the NDK, one payload for both ABIs; ran on an API 24 x86_64 emulator image | verified | +| `aarch64-linux-android` | the same payload and the same build; no execution path from an x86_64 host, because Google's emulator refuses a foreign guest | preview | +| `aarch64-ios` · `aarch64-ios-sim` · `x86_64-ios-sim` | the iPhoneOS and iPhoneSimulator SDKs ship inside Xcode and are not redistributable, so the blocker is a licence rather than a payload | planned | `verified` an image has been built **and run** for the row, qemu and wine included · `preview` it builds and links, and no emulator run has been recorded diff --git a/README.zh-CN.md b/README.zh-CN.md index c029f5435..1d2dde9f3 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -406,6 +406,10 @@ mcpp 的身份模型是两条正交轴:**工具链** = `family@version`(family | `aarch64-none-elf` · `x86_64-none-elf` | llvm 22——裸机,默认不带 C 库 2 | preview | | `thumbv7em-none-eabi` · `thumbv8m.base-none-eabi` · `thumbv8m.main-none-eabihf` | llvm 22——Cortex-M4/M7 软浮点、M23、M33F/M55F 2 | preview | | `riscv64-linux-musl` · `aarch64-linux-gnu` · `x86_64-macos` | — | planned | +| `wasm32-emscripten` | `emsdk@6.0.9` —— Emscripten 自带 sysroot 和它自己的 libc++ 模块面;`mcpp run` 用 `node` 把模块跑起来 | verified | +| `x86_64-linux-android` | `android-ndk@30.0.16248370` —— bionic 来自 NDK,一个载荷服务两个 ABI;在 API 24 的 x86_64 模拟器镜像上跑过 | verified | +| `aarch64-linux-android` | 同一个载荷、同样的构建;从 x86_64 宿主没有执行路径,因为 Google 的模拟器直接拒绝异构 guest | preview | +| `aarch64-ios` · `aarch64-ios-sim` · `x86_64-ios-sim` | iPhoneOS 与 iPhoneSimulator 的 SDK 在 Xcode 里且不可再分发,所以阻塞项是许可而不是载荷 | planned | `verified` 该行的镜像已被构建**并运行**过,qemu 与 wine 都算 · `preview` 可构建 可链接,未记录过模拟器运行 · `planned` 已登记在词表中,尚未接线 —— 面向这类目标 diff --git a/bench/matrix.json b/bench/matrix.json index 7bfcf24b6..537517ec3 100644 --- a/bench/matrix.json +++ b/bench/matrix.json @@ -1,11 +1,23 @@ { "schema": 2, "_comment": [ - "THE benchmark matrix. Read by .github/workflows/bench.yml to plan its jobs and", - "by tests/e2e/233_bench_matrix.sh to check this file against the harness's own", - "vocabulary. bench/SPEC.md explains the axes; it deliberately does not repeat", - "the cell list, because a matrix written down twice is a matrix that disagrees", - "with itself.", + "THE benchmark matrix. Read by tests/e2e/233_bench_matrix.sh, which checks it", + "against the harness's own vocabulary and against the pinned trees. bench/SPEC.md", + "explains the axes; it deliberately does not repeat the cell list, because a", + "matrix written down twice is a matrix that disagrees with itself.", + "", + "THERE IS NO .github/workflows/bench.yml IN THIS REPOSITORY. This comment used", + "to name one, and so did 233's own justification for printing a note instead of", + "failing -- which is why the hub/body existence check ran in zero CI jobs from", + "the day it was written, and why the three mcpp cells below carried a path from", + "the CURRENT source layout while naming a HISTORICAL tree (#599). 233 now runs", + "with the submodules checked out in ci-linux-e2e.yml and fails rather than notes", + "when they are absent on a runner. Running the benchmark itself is still manual.", + "", + "A `hub` OR `body` PATH BELONGS TO THE PIN, NOT TO THIS REPOSITORY. Each", + "workload is a historical commit, so a path that is correct in the current tree", + "is wrong for it by construction and the next module move breaks it again. When", + "a pin is bumped, re-read these two paths out of the new tree.", "", "A cell is one CI job. Inside it the harness sweeps every engine x variant x", "scenario, so those axes are per-cell lists rather than more jobs: they share a", @@ -133,7 +145,7 @@ "engines": "mcpp,mcpp[schedule=on],cmake,xmake", "variants": "modules", "scenarios": "cold,noop,touch-hub,edit-body,edit-comment", - "hub": "modules/platform/src/platform.cppm", + "hub": "src/platform/platform.cppm", "body": "src/version_req.cppm", "note": "KNOWN GAP: cmake cannot GENERATE this project on the runner. It configures (the compiler probe passes) and then fails with `CMake Error: the \"CXX_MODULE_STD\" property ... requires that the \"__CMAKE::CXX23\" target exist, but it was not provided by the toolchain. Reason: Only `libstdc++` is supported`. Everything checkable from outside has been checked and every one of them AGREES with a developer box where the same arm configures, generates and builds: same cmake (the very same `xim-x-cmake/4.0.2` xlings payload, not merely the same version), same `xim-x-gcc/16.1.0`, `libstdc++.modules.json` present on the runner, both sources it names (`std.cc`, `std.compat.cc`) present on the runner, and the runner's exact flag shape (`-B/lib -L/lib`, the branch this machine does not normally take) reproduced locally via a fake MCPP_HOME with no subos — it works there. What is left is inside CMake's own detection. The arm's CMakeConfigureLog is now attached to the cell log on failure so the next look starts from CMake's own record rather than another hypothesis. BASELINE MOVED to the released mcpp for this cell, exactly as the xlings cells do: normalising against an engine that never produced a binary prints bare seconds under a heading that says `relative to`. xmake and both mcpp arms stay measured here. The published cmake numbers in bench/README were taken on a machine where this arm works.", "buildfiles": "mcpp", @@ -147,7 +159,7 @@ "engines": "mcpp,mcpp[schedule=on],cmake,xmake", "variants": "modules", "scenarios": "cold,noop,touch-hub,edit-body,edit-comment", - "hub": "modules/platform/src/platform.cppm", + "hub": "src/platform/platform.cppm", "body": "src/version_req.cppm", "buildfiles": "mcpp", "allow_failed": "xmake", @@ -160,7 +172,7 @@ "engines": "mcpp,mcpp[schedule=on],cmake,xmake", "variants": "modules", "scenarios": "cold,noop,touch-hub,edit-body,edit-comment", - "hub": "modules/platform/src/platform.cppm", + "hub": "src/platform/platform.cppm", "body": "src/version_req.cppm", "buildfiles": "mcpp", "allow_failed": "cmake,xmake", diff --git a/docs/03-examples.md b/docs/03-examples.md index 98e26828a..2d0dd2346 100644 --- a/docs/03-examples.md +++ b/docs/03-examples.md @@ -82,6 +82,7 @@ the map; the table below is what each sub-example adds. |---|---| | [`08-build-rules`](../examples/08-build-rules/) | two rule packages and a project using both; `host-module = true`, `mcpp::action` with `role = "check"` | | [`12-a-new-device-language`](../examples/12-a-new-device-language/) | `device_extensions` and `rule_module`: a rule package teaching mcpp a language the engine has never heard of, whose compiler is a package built through `tools = [...]` for the build machine | +| [`13-platform-targets`](../examples/13-platform-targets/) | one source and no `cfg`, built for Linux, WebAssembly and both Android ABIs by changing only `--target`; `min_api_level` as the project's own decision, and a capability pin that cannot be overridden | [31 — Authoring a Rule Package](31-authoring-a-rule-package.md) is the reference these two illustrate. diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 6f3945e0d..6062fcc57 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -474,11 +474,32 @@ bmi_schedule = "off" # auto (default, = off) | on | off machine doing the build**, never frozen into the manifest: it takes the physical core count on a heterogeneous CPU (a 13900K is 8 P-cores + 16 E-cores, so its 32 threads are not 32 equal workers) and clamps that by free memory, because a -single module interface compile peaks at 0.5–1.0 GB. Precedence is -`--jobs` / `MCPP_JOBS`> this key> the backend's own default. A malformed value +single module interface compile peaks at 0.5–1.0 GB. A malformed value is **reported, never silently treated as the default** — a typo that quietly restores the default is a build slower than requested, with no indication why. +Precedence, and each level describes a different thing: + +| level | scope | +|---|---| +| `--jobs` / `MCPP_JOBS` | this invocation | +| `[build] jobs` (this key) | this project | +| `[build] default_jobs` in `~/.mcpp/config.toml` | **this machine** | +| absent, or `0` | say nothing, and leave the backend's own default | + +The per-machine key is the only one of the three that can hold a machine fact. +`--jobs` must be repeated on every invocation; this key is per-package and +`[workspace.build]` does not inherit it, so a seven-member workspace would carry +the number seven times and commit one developer's memory limit to the +repository. `default_jobs = 0` is what mcpp writes into a fresh config and means +absent. + +`default_jobs` **also bounds `mcpp test`'s concurrency**, where the fallback +when it is absent is the whole machine rather than a backend default. A test +runner at ten concurrent processes has the same memory shape as a compile at +ten, so a machine-wide number applies to both. This is stated because one key +with two behaviours has to be. + `bmi_schedule` decides when importers are unblocked. | value | | @@ -664,6 +685,44 @@ linkage = "static" Moved to [22 — The Target Side](22-target-side.md). +### 2.7.3 `min_api_level` — the oldest OS release the artifact must run on + +```toml +[target.aarch64-linux-android] +min_api_level = 24 +``` + +Android's own term is **API level**, and this is the minimum — the quantity the +NDK's CMake toolchain documents `ANDROID_PLATFORM` as carrying ("the minimum +API level supported by the application or library"), and which corresponds to +Gradle's `minSdk`. + +**It is a project decision and not a property of the toolchain.** One NDK +serves a range of levels, so naming `android-ndk@` does not pin one. + +**It reaches the compiler and not the identity.** The canonical triple stays +`aarch64-linux-android`, which is what names the output directory, `cfg(env = +"android")` and the packed ABI tag; the level is fused onto the triple the +compiler is given: + +| | | +|---|---| +| canonical triple | `aarch64-linux-android` | +| what clang receives | `aarch64-unknown-linux-android24` | +| build fingerprint | carries the level | + +The fingerprint is not optional: the level selects which bionic symbols are +visible, so two levels are two ABIs and must never share a build directory. + +Unset is legal and means the NDK's own default, which is what +`clang -target aarch64-linux-android` normalises to. + +This is the same mechanism `macos_deployment_target` (§ above) uses, and the +two are named in their own platforms' words rather than in a shared +abstraction. Both answer one question: the oldest OS release the artifact has +to run on. + + ### 2.7.2 Bare metal (`os = none`) — freestanding targets `riscv64-none-elf` and `riscv32-none-elf` are targets with no operating system diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index d99bb095e..b008014db 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -140,6 +140,7 @@ mcpp pack --mode self-contained # alias: --mode bundle-all mcpp pack --target x86_64-linux-musl # equivalent to --mode static mcpp pack --target aarch64-linux-musl # ARM64 equivalent mcpp pack --format dir # output as a directory, no tarball +mcpp pack --format appimage # a format a package in the graph provides mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path mcpp pack --profile dev # build with a different profile (default: release) @@ -147,6 +148,47 @@ mcpp pack --no-strip # ship the artifacts as built mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/ ``` +### `--format` owns one axis, and the engine owns two of its values + +`tar` and `dir` answer the same question `msi` and `appimage` answer — what +shape does the output take — so they are values of one flag rather than the +beginning of a second one. The split between what the engine holds and what a +package holds is: + +> **`mcpp pack` owns the mechanism and the one universal format. Every other +> format lives in a package, and `mcpp pack` dispatches to it.** + +The universal format is what it already produces: an archive that extracts and +runs. It is universal in the only sense that matters here — it needs no +knowledge of anyone else's release. Everything past it does. dpkg's control +fields, AppImage's runtime, WiX's schema, Apple's notarisation, Android's +signing scheme: each one bound into the engine would couple an mcpp release to +a release mcpp does not control. The same argument the project already made for +languages, where Slang is supported without being named in the engine. + +So the value set is open (mcpp 2026年9月11日.1+). `--format ` finds the +package in the resolved graph that declares `` and hands it the staged +tree; an unknown value names what *is* available rather than a fixed list: + +``` +error: unknown --format 'bogus'. + available in this build: tar, dir, appimage + A format past `tar` and `dir` comes from a package in the resolved graph, which declares + it with `mcpp::provides_pack_format("")` in its build program. Add the package + that provides 'bogus' to [build-dependencies] and activate its feature. +``` + +The refusal arrives before anything is compiled. Writing such a package is +[Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111); +the engine's three additions are a staged tree an `artifact` action can consume, +the rest of `[package]` in the build program, and this dispatch. Each is +format-neutral, which is the test for whether something belongs in the engine +at all. + +A dispatched format applies to a **program** target. A library package ships an +interface plus prebuilt binaries per triple and has no single staged tree, so +`mcpp pack --format ` is refused rather than ignored. + When `-o` is given a bare filename, the output is placed under `target/dist/`; when it includes a directory (relative or absolute), the literal path is used. @@ -421,8 +463,13 @@ macOS **program** bundling (the Mach-O dependency closure, via `otool -L` / `LC_LOAD_DYLIB`, and `install_name_tool` for relocation) is still on the roadmap; until it lands `mcpp pack ` refuses on that format rather than producing something that only looks like a bundle. Windows DLL bundling beyond -the current `.zip`, and distribution formats such as `.deb` / `.rpm` / AppImage, -are also on the roadmap. This document evolves alongside the -`mcpp pack` implementation; for the latest options, refer to -`mcpp pack --help`. +the current `.zip` is also on the roadmap. + +Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` are **not** on +this list, and that is a decision rather than an omission: they live in +packages and reach the user through `--format `, for the reason the +section above gives. Nothing further needs to join `[pack]`'s built-in modes. + +This document evolves alongside the `mcpp pack` implementation; for the latest +options, refer to `mcpp pack --help`. diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index 0756c2152..30906bbe8 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -45,7 +45,7 @@ Subsequent builds do not trigger this process again. Two orthogonal axes name everything: -- **toolchain** = `family@version`, family ∈ `gcc | llvm | msvc` — *who compiles* +- **toolchain** = `family@version`, family ∈ `gcc | llvm | msvc | emsdk | android-ndk` — *who compiles* - **target** = a triple `arch-os[-env]` (e.g. `x86_64-linux-musl`, `x86_64-windows-gnu`, `aarch64-macos`) — *what it produces for* @@ -491,6 +491,112 @@ is built per project and cl bakes `_MSVC_MT`/`_MSVC_MD` into it, so a per-role override (`cxx_runtime = { tests = ... }`) is refused with a message saying so rather than producing a module mismatch inside the ucrt headers. +## SDK Toolchains (`emsdk`, `android-ndk`) + +Two of the five toolchain spellings name an **SDK** rather than a bare +compiler: `emsdk` and `android-ndk`. Their compiler *is* clang -- so they are +not a separate compiler family, and mcpp does not pretend they are -- but the +archive brings its own sysroot, its own C library and, for both of these, its +own generated `std` module surface. That difference is what the rest of this +section is about. + +### Nothing has to be declared + +A target row names its own payload, and that pin is the default. Neither of +these needs a line in `mcpp.toml`: + +```bash +mcpp build --target wasm32-emscripten # resolves emsdk@6.0.9 +mcpp build --target aarch64-linux-android # resolves android-ndk@30.0.16248370 +``` + +The payload is **installed on demand** the first time a target needs it, the +same way a gcc or llvm payload is. `mcpp toolchain list` shows the pin beside +the row, and the build reports which archive answered: + +``` +Resolved emsdk@6.0.9 → wasm32-emscripten → .../xim-x-emsdk/6.0.9/emscripten/em++ +Resolved android-ndk@30.0.16248370 → aarch64-linux-android → .../prebuilt/linux-x86_64/bin/clang++ +``` + +### Declaring one anyway + +The ordinary per-target key works, and naming the row's own payload is always +accepted: + +```toml +[target.aarch64-linux-android] +toolchain = "android-ndk@30.0.16248370" + +[target.wasm32-emscripten] +toolchain = "emsdk@6.0.9" +``` + +Use it to pin a version across machines, or to opt into a payload newer than +the row's convention. The **version** is free -- anything the index publishes +resolves -- so this is how a project moves ahead of, or stays behind, the +default. + +### What cannot be overridden, and why + +For these rows the pin is a **capability** rather than a convention: it is not +mcpp's preference among several payloads that could serve the target, it is the +only thing that can. So the payload NAME is fixed while the version is open: + +```toml +[target.aarch64-linux-android] +toolchain = "llvm@22.1.8" # refused +``` + +``` +error: target 'aarch64-linux-android' cannot be emitted by 'llvm@22.1.8'. + An Android target needs bionic, not just an aarch64 or x86_64 back end: + its headers, its per-API-level stubs and its loader path are inside the + NDK, and no package adds them to another compiler. +``` + +The refusal is not about code generation. A stock clang emits aarch64 ELF +perfectly well; what it cannot supply is the SYSTEM. Saying so at the +declaration is better than resolving llvm and failing deep inside the build, +which is what happened before this gate existed -- first `'__config' file not +found`, then `Unversioned target triples are not supported!` from bionic's own +header, neither of them naming the toolchain that could not serve the row. + +`wasm32-emscripten` refuses on the same rule with its own sentence: nothing but +Emscripten emits WebAssembly. + +### What belongs to the project instead + +The toolchain is the SDK's; the **deployment floor** is the project's, and it +has its own key per platform -- see +[04 — mcpp.toml](04-mcpp-toml.md) §2.7.3: + +```toml +[target.aarch64-linux-android] +min_api_level = 24 # Android +``` + +```toml +[package] +macos_deployment_target = "14.0" # Apple +``` + +One NDK serves a range of API levels, so the level is a project decision and +naming `android-ndk@` does not pin one. Left out, mcpp reads the floor +the NDK itself declares in `meta/platforms.json`. + +### Running what they produce + +Neither the emulator nor a device is part of the toolchain axis. `mcpp run` +executes a wasm module directly, because Emscripten's output is a program +`node` can run. For a target whose artifact runs elsewhere, the `runner` key is +an argv prefix and the session belongs to a package rather than to the engine: + +```toml +[target.x86_64-linux-android] +runner = ["adb-run"] # a program from xim:android-platform-tools +``` + ## Project-Level Version Pinning If a project needs to pin a specific version rather than rely on the global default, declare it in the project's `mcpp.toml`: @@ -776,7 +882,10 @@ and refuses. **Scope.** The contract governs the C++ runtime only. Static **libc** is a separate axis (`linkage = "static"` / `--static`, e.g. a musl target), and the deployment -floor is a third (`macos_deployment_target`). Also, `host-coupled` means mcpp adds +floor is a third — `macos_deployment_target` in `[package]` for Apple targets, +`min_api_level` under `[target.
]` for Android. Both feed one parameter +of `llvm_triple()` and one slot in the build fingerprint, because a target is +either Apple or Android and the two answer the same question. Also, `host-coupled` means mcpp adds nothing to embed a C++ runtime; it does not strip the toolchain rpath the link carries for other reasons, so on ELF such an artifact may still find the toolchain's libraries first. diff --git a/docs/21-the-target-triple.md b/docs/21-the-target-triple.md index b6a74e998..9b563f77d 100644 --- a/docs/21-the-target-triple.md +++ b/docs/21-the-target-triple.md @@ -38,19 +38,34 @@ and the build reports what it resolved. | Segment | Content | Example | |---|---|---| -| `arch` | instruction set | `x86_64`, `aarch64`, `riscv64` | -| `os` | operating system, or `none` | `linux`, `windows`, `macos`, `none` | -| `env` | see below — it is a different axis per platform | `gnu`, `musl`, `msvc`, `elf` | +| `arch` | instruction set | `x86_64`, `aarch64`, `riscv64`, `wasm32` | +| `os` | operating system, or `none` | `linux`, `windows`, `macos`, `ios`, `emscripten`, `none` | +| `env` | see below — it is a different axis per platform | `gnu`, `musl`, `msvc`, `android`, `elf` | The third segment is the one that repays attention, because it does not name the same kind of thing everywhere: | Platform | `env` names | Values | |---|---|---| -| `linux` | the **C library** | `gnu` (glibc), `musl` | +| `linux` | the **C library** | `gnu` (glibc), `musl`, `android` (bionic) | | `windows` | the **object ABI** | `gnu` (Itanium C++ ABI), `msvc` (Microsoft's) | | `none` | the **object format** | `elf` | -| `macos` | nothing; the platform carries no segment | — | +| `ios` | the **device or the simulator** | `sim` (simulator), absent (device) | +| `macos`, `emscripten` | nothing; the platform carries no segment | — | + +`sim` is the one Apple row with a segment, and it names neither a C library nor +an ABI: a simulator build has its own SDK (`iPhoneSimulator.sdk`), produces its +own object, and takes `-mios-simulator-version-min` where the device takes +`-miphoneos-version-min`. Two targets, so two identities. Apple spells it with +a trailing `-simulator` on the OS segment and Rust with `aarch64-apple-ios-sim`; +both spellings parse here, and both canonicalise to `aarch64-ios-sim`. + +`android` is a **C library** and therefore sits where `musl` sits, on a `linux` +OS. That placement is the whole of the modelling decision: the kernel *is* +Linux, so ELF, the `unix` family and `nasm -f elf64` are already right, and an +`os = "android"` would have made every one of them wrong by default and needed +a new answer at each site. What differs from `gnu` is bionic, the loader path +and the SDK — which is exactly what an `env` value is for. On Windows the segment is frequently misread, because the word `gnu` suggests a C library that is not there. Measured on an artefact built for @@ -68,6 +83,30 @@ openkal. `gnu` is LLVM's label for the non-MSVC ABI, inherited from MinGW, and clang requires that spelling to select the right internal toolchain. mcpp cannot rename it. +### The object format is an axis, not a derivation + +A triple's binary format used to be nothing at all: it was re-derived from `os` +wherever it was needed. `is_pe()` asked `os == "windows"`, artifact naming asked +again, the packer asked a third time. That is affordable while the answer has +two values. + +`wasm32` is the first target in mcpp's vocabulary whose format is neither, and a +third value turns those derivations into an addition **at every such site** — and +a site that is missed does not fail. It silently answers ELF, because ELF is +what every `else` branch in the tree assumes. So the format is now one answer: + +| target | format | +|---|---| +| `x86_64-linux-gnu`, `aarch64-linux-android`, `riscv64-none-elf` | ELF | +| `aarch64-macos`, `aarch64-ios` | Mach-O | +| `x86_64-windows-gnu`, `x86_64-windows-msvc` | PE | +| `wasm32-emscripten` | wasm | + +It is **not** the same question as "is there an operating system to link +against". A bare-metal RISC-V image is ELF with no OS; a wasm module has an +OS-like layer (Emscripten's POSIX emulation) and is not ELF. Merging the two +axes is the mistake this replaces. + ## Declining The Third Segment `-` is a complete target on every platform: @@ -444,6 +483,14 @@ other's rows. | `thumbv8m.base-none-eabi` | preview | `llvm@22.1.8` | payload | payload | payload | payload | | `thumbv8m.main-none-eabi` | verified | `llvm@22.1.8` | payload | payload | payload | payload | | `thumbv8m.main-none-eabihf` | preview | `llvm@22.1.8` | payload | payload | payload | payload | +| `armv7a-none-eabi` | verified | `llvm@22.1.8` | payload | payload | payload | payload | +| `armv7a-none-eabihf` | verified | `llvm@22.1.8` | payload | payload | payload | payload | +| `aarch64-linux-android` | preview | `android-ndk@30.0.16248370` | payload | payload | payload | — | +| `x86_64-linux-android` | verified | `android-ndk@30.0.16248370` | payload | payload | payload | — | +| `aarch64-ios` | planned | — | planned | planned | planned | planned | +| `aarch64-ios-sim` | planned | — | planned | planned | planned | planned | +| `x86_64-ios-sim` | planned | — | planned | planned | planned | planned | +| `wasm32-emscripten` | verified | `emsdk@6.0.9` | payload | payload | payload | payload | `payload` a toolchain payload here produces it · `graph` no payload, but a dependency can supply the system · `system` located on the machine, not @@ -461,12 +508,31 @@ installed by mcpp · `SDK` the platform's own · `—` unreachable from this hos | `x86_64-windows-musl` | Windows by payload; anywhere by graph | no gcc emits PE+musl, and LLVM cannot spell the triple | | `aarch64-macos` | macOS | the SDK is the machine's | | `*-none-elf` | every host | clang and lld are cross-compilers by construction | +| `wasm32-emscripten`, `*-linux-android` | every host | the SDK ships its own sysroot and upstream publishes it per host; one archive serves every guest arch | **A `—` is about payloads, not about possibility.** `host_can_serve` answers "does a payload here produce it", and a dependency graph can supply the system instead — which is why `x86_64-windows-musl` reads `via dependency graph` on Linux and produces a real PE32+ there. +**And for the two Android rows the `—` on Windows is the INDEX's answer, so +`mcpp toolchain list` still shows them there.** Google publishes a Windows NDK +and it downloads; what it does not contain is the libc++ module surface +(measured: no `std.cppm` and no `std/*.inc`, against 110 on the other two +hosts), so `xim:android-ndk` declares no Windows table — an entry that can +never serve a module-first build is worse than none. The engine does not encode +that: which hosts an index serves changes without an engine release, and a +constant stating it here is what the wasm row's own history shows going stale. +A Windows user therefore sees the row, the pin resolves, and xim refuses with +`no payload for this platform` before anything is fetched, naming the package. + +**The two Android rows differ in tier because one of them was run.** An +x86_64 Android artefact executes on the platform's own emulator, and a +`verified` row means exactly that was done. The device row builds identically +and has no execution path from an x86_64 host: the emulator refuses a foreign +guest (`QEMU2 emulator does not support arm64 CPU architecture`), so it needs +an arm64 host or the qemu-user route. + ### And CI measures every one of them [`ci-target-matrix.yml`](../.github/workflows/ci-target-matrix.yml) runs on all diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index e48a22a15..6f8d92b5e 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -178,6 +178,91 @@ the object ABI on Windows, the object format where there is no operating system — and one value records which, rather than a boolean recording only whether the first case holds. +## Android, Web And iOS Under This Model + +The three platforms mcpp added target rows for in 2026年9月11日.3 are not one +question. What decides each is where its implementation would have to sit +relative to a C library, and the answers are different. + +### Android shares the Linux implementation, unchanged + +`openkal-linux` is written on the Linux kernel's own system-call interface and +borrows nothing from any C library — that is what lets it be placed beneath one. +Android's kernel **is** Linux, the system-call ABI for a given architecture is +the same, and `src/sys.h` dispatches on `__x86_64__` / `__aarch64__`, which is +the architecture rather than the operating system. Nothing in it is glibc's or +bionic's. + +So a portable program needs no new line. `cfg(os = "linux")` is **true for an +Android triple**, because Android is an `env` value on a `linux` OS — the +modelling decision [21 — The Target Triple](21-the-target-triple.md) records — +and the implementation is selected by the line a Linux consumer already writes: + +```toml +[target.'cfg(os = "linux")'.dependencies] +openkal-linux = "0.12.0" +``` + +Measured 2026年09月11日, a program written against openkal and nothing else — no C +library, no `import std`: + +``` +mcpp build --target x86_64-linux-android + kernel-abi openkal (openkal-linux@0.12.0, graph) + -> ELF 64-bit LSB pie, x86-64, interpreter /system/bin/linker64 + +mcpp build --target aarch64-linux-android + -> ELF 64-bit LSB pie, ARM aarch64, same interpreter +``` + +and the x86_64 artifact, pushed to an API 24 emulator image and executed: + +``` +openkal: 1-2-3 exit 0 +``` + +`openkal-linux` itself also compiles for both Android targets unchanged, which +is the weaker claim of the two and is worth stating separately: the first says +the implementation builds, the second says a program over it runs. + +### iOS would share the macOS implementation, and that cannot be claimed yet + +The same argument applies on Apple's side — iOS and macOS share the Darwin +kernel, and `openkal-macos` is arch-dispatched the same way — but the argument +is not evidence. The iPhoneOS and iPhoneSimulator SDKs ship inside Xcode and +are not redistributable, so the `aarch64-ios` and `*-ios-sim` rows are +`planned`: there is nothing to build against and therefore nothing to run. +Declaring support on a structural argument alone is the shape this ecosystem has +paid for before — a package present in an index is not a package that builds a +real project — so these rows claim nothing until an SDK is reachable. + +### Web needs a new implementation, and a different one + +Emscripten is the one of the three that changes the model rather than extending +it. There is no kernel and there are no system calls to issue: Emscripten +supplies its own C library over a JavaScript host. An openkal implementation for +it therefore cannot be written the way `openkal-linux` is — beneath a C library +— and would have to sit **above** one. The specification permits exactly that +("an implementation may be built upon a C library, beneath one, or without +one"), so this is new software rather than a sharing decision, and it is the one +of the three that is neither done nor blocked. + +Until it exists, `wasm32-emscripten` is served the ordinary way: by a payload. +`xim:emsdk` ships the compiler, the sysroot and a libc++ module surface, so a +program that uses `import std` builds and runs for the Web today without openkal +being involved at all — which is what the row's `verified` tier records. + +### The table + +| platform | implementation | status | +|---|---|---| +| Linux (glibc, musl) | `openkal-linux` | the reference implementation | +| Android (both ABIs) | `openkal-linux`, unchanged | builds; a program over it ran on an emulator | +| macOS | `openkal-macos` | on the macOS system-call surface | +| iOS, iOS simulator | `openkal-macos` would serve it | blocked: the SDK is not redistributable | +| Windows | `openkal-windows` | on Win32 and the object manager | +| Web (Emscripten) | none | needs an implementation written ABOVE a C library | + ## Bare Metal A target with no operating system is the same model with the platform layer diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 56e7b9195..43844a8ab 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -463,6 +463,12 @@ attach: | `object` | join the **link** set | the link edge consumes them | a resource compiler, `objcopy` embedding a blob, a generated `.def`, a pre-built `.o` | | `artifact` | a new file | its *inputs* are link outputs, so it runs after the link | codesign, packaging, size budgets | +`artifact` is also the only role that may name `${mcpp.stage_dir}` — see +[Producing a distributable](#producing-a-distributable-pack_format--stage_dir-20269111) +below. The other three run before or alongside the link, so there is nothing +staged for them to read, and mcpp refuses the placeholder rather than expanding +it to a path that happens to exist. + No phase machinery is involved. `object` and `artifact` are sequenced by ninja's own file dependencies — which is also why an `artifact` action cannot double-apply itself the way a naive "post-build hook" would. `source` and a @@ -553,6 +559,89 @@ scan agrees with what the generator will emit — the same assertion-plus- verification trade `[modules].scan_overrides` makes, and the compiler's own P1689 output checks it at build time. +### Producing a distributable: `pack_format` / `stage_dir` (2026年9月11日.1+) + +An `.msi`, a `.deb`, an AppImage and a signed `.app` are none of the four roles' +usual work and all of them are `artifact`: each consumes **link outputs** and +produces something a user installs. What the engine adds for them is a +mechanism and no format at all. + +`mcpp pack --format ` resolves `` through the resolved graph, the +same way `--target` reaches a triple the engine did not have to know +individually. `tar` and `dir` remain the archive shapes `mcpp pack` owns; +everything past them comes from a package. + +A provider has two halves, and they must not be merged: + +```cpp +import mcpp; +#include +#include + +int main() { + // Half one, unconditional. + mcpp::provides_pack_format("appimage"); + + // Half two, conditional. + if (std::string_view(mcpp::pack_format()) != "appimage") return 0; + + const std::string out = std::string(mcpp::out_dir()) + "/app.AppImage"; + mcpp::action a; + a.id = "appimage"; + a.role = "artifact"; + a.arg(tool).arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +``` + +**Declare unconditionally, submit conditionally.** The declaration is what lets +the engine answer a question the requesting build cannot: `--format bogus` +names what *is* available, and `--help` says "any format the resolved graph +provides". Both read the set collected from a pass that asked for nothing. A +member that declared only when asked still works for its author — they always +pass their own format — and makes the set unknowable for everyone else. mcpp +refuses a format nothing submitted for, rather than reporting a pack that +produced no package. + +**`mcpp pack --format ` prepares twice.** An `artifact` action is a ninja +edge and the staged tree is produced by mcpp *after* the link, so the tree +cannot be an input of the pass that built it. The first pass collects the +declarations and refuses an unknown format before anything is compiled; the +build and the staging then happen; the second pass sets `pack_format` and +`pack_stage_dir` and builds the edge the provider submits. Nothing in the +second pass is re-derived — the triple and the staged path are what the first +pass and the staging already answered. + +**The staged tree is a bundle, not a root filesystem.** It is what +`--mode vendored` means: `bin/`, `lib/`, relocatable, rooted anywhere, after the +strip policy, the debug-symbol split and `include`/`exclude`. An AppImage, a +`.app` and an `.msi` want it as it stands. A format that wants an FHS tree +(`.deb`, `.rpm`) owns the re-layout, because which directory a file belongs in +is that format's knowledge and not the engine's. + +`mcpp pack --format dir` writes the same tree to a path and stops, which is how +a person inspects what a member will be handed. + +**`mcpp pack` reports the artifact actions the request introduced.** An action +present whether or not a format was asked for — a codesign stamp, a size budget +— is not the distributable, and naming one would be a wrong answer that looks +like a right one. Nothing about the criterion is a property of the member: a +format that packages one named program and never reads the staged tree is +recognised exactly as one that consumes the whole closure. A format nothing +submitted for is refused by name. + +**An action that names `${mcpp.stage_dir}` gains a dependency on the tree's +manifest.** mcpp writes `.stage-manifest` — a sibling, never a +member, so it does not travel inside anyone's installer — listing each staged +file's size and relative path. The dependency is added by the engine because +the use implies it: without it the edge is dirty only when a link output +changes, and a closure that grew a dependency's shared library while the +program's own bytes did not would leave the previous distributable in place, +reported as up to date. + Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: @@ -562,6 +651,7 @@ none to rely on), and the only interpolations are a closed set: | `${mcpp.bin_dir}` | where produced binaries land | | `${mcpp.compile_db}` | path to `compile_commands.json` (what clang-tidy's `-p` wants) | | `${mcpp.target_file:}` | the built file of target `` | +| `${mcpp.stage_dir}` *(2026年9月11日.1+)* | the tree `mcpp pack` staged, absolute. `artifact` role only, and only under `mcpp pack --format ` | The raw stdout protocol above remains the low-level substrate; `import mcpp;` is the typed layer over it. @@ -671,6 +761,13 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_LANGUAGE_MODULES` *(2026年9月7日.1+)* | -- | `1` when the declaring package sets `[language] modules`, `0` otherwise. A rule that GENERATES a consumer-facing declaration reads it to choose between a module interface and a header, so a project states that once and never again. An older engine leaves it absent, which a rule reads as `0` -- the behaviour every consumer had before the variable existed | | `MCPP_PKG_NAME` *(2026年9月7日.1+)* | -- | The `[package] name` of the package this program builds. Every name a rule generates is derived from it: the module a consumer imports, the namespace the accessors sit in, the symbols in a generated header. Before it existed the closest available answer was the leaf of `MCPP_MANIFEST_DIR`, which is a directory name -- so a package named `vulkan-saxpy` in a directory named `app` generated `app.shaders`, and every `/app/` in a workspace claimed the same module. Absent under an older engine, which a rule reads as a signal to keep its previous derivation | | `MCPP_PKG_NAMESPACE` *(2026年9月7日.1+)* | -- | The `[package] namespace`. Empty when the package declares none. A rule that must produce a name unique across an index uses the pair rather than the name alone, because package identity is `(namespace, name)` | +| `MCPP_PKG_VERSION` *(2026年9月11日.1+)* | `mcpp::package_version()` | The `[package] version`. Every installer format states a version; without this the project had to restate it in the member's own options, where the copy drifts from `[package]` with nothing able to detect it | +| `MCPP_PKG_DESCRIPTION` *(2026年9月11日.1+)* | `mcpp::package_description()` | The `[package] description`. Empty when the package declares none | +| `MCPP_PKG_LICENSE` *(2026年9月11日.1+)* | `mcpp::package_license()` | The `[package] license` | +| `MCPP_PKG_AUTHORS` *(2026年9月11日.1+)* | `mcpp::package_authors()` | The `[package] authors`, joined with `;`. Not `,`: an author entry is conventionally `Name ` and a name may carry a comma, so a comma-joined list cannot be split back into the entries it was made from | +| `MCPP_PKG_REPO` *(2026年9月11日.1+)* | `mcpp::package_repo()` | The `[package] repo` | +| `MCPP_PACK_FORMAT` *(2026年9月11日.1+)* | `mcpp::pack_format()` | The `--format` value of the `mcpp pack` pass this program is part of; empty for every ordinary build. The empty value is the one that carries the meaning — a member gates its submission on this, so `mcpp build` has the graph it always had | +| `MCPP_PACK_STAGE_DIR` *(2026年9月11日.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | | `MCPP_DEVICE_SOURCES` *(2026年9月5日.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, ...) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable scratch/output dir owned by mcpp | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | @@ -786,14 +883,30 @@ and warns when the two disagree — Nothing breaks; the name claims an origin the package does not have. A rule outside the project picks its own prefix. -**A tool is not a rule.** A rule states how a translation unit is compiled by a -compiler mcpp does not drive: it submits an action and the engine schedules it. -A tool states something the build program needs that no compiler performs, and -does it while the program runs. `mcpp.tools.embed` (feature `tools-embed`, +**A tool is not a rule, and a distributable is neither.** Three kinds of work, +and the member's prefix says which of the three questions it answers: + +| Prefix | Question | Compiles a TU | Runs in the build program | +|---|---|---|---| +| `rules-*` | how is this translation unit compiled | yes | no | +| `tools-*` | what does the build program need to do itself | no | yes | +| `dist-*` *(2026年9月11日.1+)* | what comes out of the link, and in what form a user installs it | no | no | + +A rule states how a translation unit is compiled by a compiler mcpp does not +drive: it submits an action and the engine schedules it. A tool states +something the build program needs that no compiler performs, and does it while +the program runs. `mcpp.tools.embed` (feature `tools-embed`, mcpp 2026年9月5日.4+) is the first: it writes a data file into a header the program compiles in, as a byte array or a 32-bit word array, and rewrites nothing when the content is unchanged, so calling it unconditionally costs no rebuild. +A `dist-*` member fits neither definition. It does not compile a translation +unit and it does not run while the build program runs: it consumes link outputs +and produces something a user installs, through an `artifact` action and +`mcpp pack --format `. The prefix matters because the taxonomy is +load-bearing — a consumer reading `rules-wix` would expect a compiler and a +translation unit, and there is neither. + `examples/09-heterogeneous/cuda` and `examples/09-heterogeneous/vulkan` consume `mcpp.rules.cuda` and `mcpp.rules.spirv` from `mcpp:plugins`, the way any project does. diff --git a/docs/31-authoring-a-rule-package.md b/docs/31-authoring-a-rule-package.md index dd770f425..92ffd6c9b 100644 --- a/docs/31-authoring-a-rule-package.md +++ b/docs/31-authoring-a-rule-package.md @@ -301,6 +301,61 @@ that declares the edge under another key can supply it, and refuses with a message naming what it looked for rather than running a command with an empty path. +## Authoring a distribution member (mcpp 2026年9月11日.1+) + +A `dist-*` member is neither a rule nor a tool. It does not compile a +translation unit and it does not do its work while the build program runs: it +consumes **link outputs** and produces something a user installs — an `.msi`, a +`.deb`, an AppImage, a signed `.app`. The mechanism is an `artifact` action and +`mcpp pack --format `, documented in +[30 — Producing a distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111). + +Everything in this chapter applies unchanged. Six things bind harder here, and +each is a mistake this category makes and a rule does not. + +**Declare unconditionally, submit conditionally.** `provides_pack_format` is +what the engine reads to answer "which formats does this graph provide", on a +build that asked for none. A member that declares only when asked works for its +author, who always passes their own format, and makes the set unknowable for +everyone else. + +**Expose a plan and a submit.** A distributable is the last thing before a +user's hands, so it is the most likely part of a build to need a +project-specific edit: a different compression level, one extra file, a second +signature. `generate_all(opt)` being exactly `submit(plan_all(opt))` is what +keeps such an edit from becoming a reimplementation of the member. + +**Name the input; do not harvest a directory.** A path that resolves to nothing +is silent, and a named input that is missing is an error. Measured: a WiX action +that bound a directory and harvested it produced a **valid, empty, 52 KB +installer with no diagnostic** when the path resolved to nothing on Windows. +`${mcpp.target_file:}` is the mechanism — a build program is told neither +the triple nor the fingerprint, and an unknown target name is refused rather +than expanded to an empty path. + +**Assert a floor on the member's own output, on the success path.** Where a +member can tell that its result is empty or implausible, it must say so through +`mcpp::warning`, because stderr on a successful build is discarded. This is the +same failure the paragraph above measured, caught one layer later. + +**Declare the tool where it will be looked up.** `xpkg_dir` answers from +`MCPP_XPKG_*_DIR`, which mcpp sets for the package *being built*. A +dependency's declaration provisions the payload without making it visible to a +consumer's build program, so a member that runs a payload tool declares it +itself — and says so when the lookup returns empty, rather than pointing at a +manifest the reader does not own. + +**A build must not reach the network, and a wrapped tool may.** Measured on +`appimagetool` 1.9.1: it downloads its type-2 runtime stub from a GitHub +release on every invocation unless `--runtime-file` names a local copy. A member +that wraps such a tool has to supply the file from its declared payload. +Install time is when a download is legitimate; build time is not, and a build +that fetches is neither reproducible nor usable offline. + +**One `(name, version)` names one payload.** A member that wraps a signing or +packaging tool inherits that tool's compatibility surface, so versioning in +lock-step with the wrapped tool is legitimate and says something true. + ## Current limitations - A rule feature that is in the package's own `[features] default` does not diff --git a/docs/README.md b/docs/README.md index 09eaffcff..cb1ecc7fa 100644 --- a/docs/README.md +++ b/docs/README.md @@ -125,6 +125,7 @@ token in front of a reader to the chapter that owns it. | `[build] accel`, `[package] accelerators`, `device_extensions` | [42](42-heterogeneous-builds.md) | `[hooks]` | [09](09-commands-by-scenario.md) | | `[package] platforms`, `[build] cache` | [04](04-mcpp-toml.md) | `[targets.]`, `[profile.]` | [04](04-mcpp-toml.md) | | `runner`, `[target..runners]` | [41](41-devices.md) | `rule_module` | [31](31-authoring-a-rule-package.md) | +| `min_api_level` | [04](04-mcpp-toml.md) | `macos_deployment_target` | [04](04-mcpp-toml.md) | **Commands** diff --git a/docs/zh/03-examples.md b/docs/zh/03-examples.md index 8a76cfe5f..ff9cc3549 100644 --- a/docs/zh/03-examples.md +++ b/docs/zh/03-examples.md @@ -77,6 +77,7 @@ mcpp build && mcpp run |---|---| | [`08-build-rules`](../../examples/08-build-rules/) | 两个规则包与同时使用它们的工程;`host-module = true`、`role = "check"` 的 `mcpp::action` | | [`12-a-new-device-language`](../../examples/12-a-new-device-language/) | `device_extensions` 与 `rule_module`:规则包教会 mcpp 一门引擎从未听说过的语言,而它的编译器是一个经 `tools = [...]` 为构建机构建出来的包 | +| [`13-platform-targets`](../../examples/13-platform-targets/) | 一份源码、零个 `cfg`,只改 `--target` 就为 Linux、WebAssembly 和两个 Android ABI 构建;`min_api_level` 作为工程自己的决定,以及一条不能被覆盖的能力钉 | [31 —— 编写规则包](31-authoring-a-rule-package.md) 是这两个示例所演示内容的参考。 diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 2854bda02..47acb367f 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -413,10 +413,28 @@ bmi_schedule = "off" # auto(默认,= 关)| on | off `jobs` 是同时跑几个编译。`"auto"` **在构建这台机器上现算**,绝不冻进 manifest: 异构 CPU 上取物理核数(13900K 是 8 P-core + 16 E-core,它的 32 个线程不是 32 个 等价的工人),再按可用内存夹一次 —— 单个模块接口编译峰值 0.5–1.0 GB。 -优先级:`--jobs` / `MCPP_JOBS`> 这个键> 后端自己的默认值。写错的值会被 +写错的值会被 **明确报出来,绝不静默当成默认值** —— 一个悄悄退回默认的拼写错误,表现是 「构建莫名其妙比我要求的慢」。 +优先级,每一级描述的是不同的东西: + +| 级别 | 作用域 | +|---|---| +| `--jobs` / `MCPP_JOBS` | 这一次调用 | +| `[build] jobs`(这个键) | 这个工程 | +| `~/.mcpp/config.toml` 里的 `[build] default_jobs` | **这台机器** | +| 缺省,或 `0` | 什么都不说,交给后端自己的默认值 | + +三者里只有按机器的那个键能承载机器事实。`--jobs` 每次调用都要重说一遍;这个键 +是按包的,而 `[workspace.build]` 不继承它,所以一个七成员的 workspace 会把同一个 +数字写七遍,并把某位开发者的内存上限提交进仓库。`default_jobs = 0` 是 mcpp 写进 +新配置的值,含义是缺省。 + +`default_jobs` **同时约束 `mcpp test` 的并发**,而在那里缺省时的回落是整台机器 +而不是某个后端的默认值。十个并发测试进程与十个并发编译的内存形状一样,所以按机器 +设的数字对两者都生效。这句话写出来是因为:一个键有两种行为,必须明说。 + `bmi_schedule` 决定**导入方什么时候被解锁**。 | 值 | | @@ -576,6 +594,40 @@ linkage = "static" 已移入 [22 —— 目标侧](22-target-side.md)。 +### 2.7.3 `min_api_level` —— 产物必须能跑在多老的 OS 上 + +```toml +[target.aarch64-linux-android] +min_api_level = 24 +``` + +Android 自己的用词是 **API level**,而这里要的是它的**最小值** —— NDK 的 CMake +toolchain 把 `ANDROID_PLATFORM` 记载为「the minimum API level supported by the +application or library」,并说明它对应 Gradle 的 `minSdk`。 + +**这是工程的决定,不是工具链的属性。** 一个 NDK 服务一个级别区间,所以写 +`android-ndk@` 并不钉住某一个级别。 + +**它到达编译器,不进入身份。** 规范 triple 仍然是 `aarch64-linux-android` —— +输出目录、`cfg(env = "android")` 和打包的 ABI tag 都由它命名;级别只拼进交给 +编译器的那个 triple: + +| | | +|---|---| +| 规范 triple | `aarch64-linux-android` | +| clang 实际收到 | `aarch64-unknown-linux-android24` | +| 构建指纹 | 含级别 | + +指纹不是可选项:级别决定哪些 bionic 符号可见,所以两个级别就是两个 ABI,绝不可 +共用一个构建目录。 + +不设也合法,含义是 NDK 自己的默认级别 —— 那正是 +`clang -target aarch64-linux-android` 规范化出的形式。 + +这与 `macos_deployment_target`(见上文)是同一套机制,而两者都用各自平台的词汇 +命名,而不是一个共享抽象。它们回答同一个问题:产物必须能跑在多老的 OS 发布版上。 + + ### 2.7.2 裸机(`os = none`)—— freestanding target `riscv64-none-elf` 与 `riscv32-none-elf` 是底下没有操作系统的 target。它们不需要 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 62e5d6c18..a787f1576 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -106,6 +106,7 @@ mcpp pack --mode self-contained # 别名:--mode bundle-all mcpp pack --target x86_64-linux-musl # 等价 --mode static mcpp pack --target aarch64-linux-musl # ARM64 等价写法 mcpp pack --format dir # 输出为目录,不打包 tarball +mcpp pack --format appimage # 由图里某个包提供的格式 mcpp pack -o myapp.tar.gz # 仅文件名:落到 target/dist/myapp.tar.gz mcpp pack -o /abs/path/myapp.tar.gz # 含目录:按字面路径输出 mcpp pack --profile dev # 换一个 profile 构建(默认 release) @@ -113,6 +114,42 @@ mcpp pack --no-strip # 按构建原样发货,不剥符号 mcpp pack --debug-symbols dbg/ # 把分离出的 *.debug 写到 dbg/ ``` +### `--format` 是一个轴,引擎只拥有其中两个取值 + +`tar` 与 `dir` 回答的问题,和 `msi` 与 `appimage` 回答的问题是同一个 —— 输出取什么 +形状 —— 所以它们是一个 flag 的取值,而不是第二个 flag 的开端。引擎持有什么、包持有 +什么,分界是: + +> **`mcpp pack` 拥有机制,以及那一种通用格式。其余每一种格式都住在包里,由 +> `mcpp pack` 分派过去。** + +那种通用格式就是它已经在产出的东西:一个解开就能跑的归档。它「通用」只在这里唯一 +要紧的那个意义上 —— 它不需要知道任何别人的发布。此外的一切都需要。dpkg 的 control +字段、AppImage 的 runtime、WiX 的 schema、Apple 的公证、Android 的签名方案:其中任何 +一个被绑进引擎,都会把一次 mcpp 的发布耦合到一次 mcpp 并不控制的发布上。这与本项目 +早已为语言做过的论证是同一个 —— Slang 被支持,而引擎里没有它的名字。 + +所以取值集合是开放的(mcpp 2026年9月11日.1+)。`--format ` 在解析后的图里找到声明 +了 `` 的那个包,并把暂存树交给它;一个未知的取值会点名**当下确实可用**的那些, +而不是一份固定清单: + +``` +error: unknown --format 'bogus'. + available in this build: tar, dir, appimage + A format past `tar` and `dir` comes from a package in the resolved graph, which declares + it with `mcpp::provides_pack_format("")` in its build program. Add the package + that provides 'bogus' to [build-dependencies] and activate its feature. +``` + +这次拒绝发生在任何东西被编译之前。怎么写这样一个包,见 +[产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111); +引擎加的三样东西是:一棵 `artifact` action 可以消费的暂存树、`[package]` 的其余字段 +进入构建程序、以及这次分派本身。每一样都与格式无关 —— 而「与格式无关」正是判断某样 +东西该不该进引擎的判据。 + +被分派的格式作用于一个**程序** target。库包发的是一份接口加上每个三元组的预构建产物, +没有单独一棵暂存树,所以 `mcpp pack <库> --format ` 会被拒绝,而不是被忽略。 + `-o` 接受裸文件名时自动归到 `target/dist/`;含目录(相对或绝对) 时按字面路径输出。 @@ -355,6 +392,11 @@ force_bundle = ["libfoo.so"] # 即使命中 PEP 600 名单也强制打包 macOS **程序** bundling(Mach-O 依赖闭包,走 `otool -L` / `LC_LOAD_DYLIB`, 重定位走 `install_name_tool`)仍在规划中;在它落地之前,`mcpp pack <程序>` 会在该格式上拒绝,而不是产出一个只是看起来像 bundle 的东西。当前 `.zip` -之外的 Windows DLL 分发,以及 `.deb` / `.rpm` / AppImage 等格式,同样在规划中。本文档随 `mcpp pack` 实现演进,最新选项以 -`mcpp pack --help` 为准。 +之外的 Windows DLL 分发,同样在规划中。 + +`.deb`、`.rpm`、AppImage、`.msi` 这些分发格式**不在**这份清单上,而这是一个决定而不是 +一处遗漏:它们住在包里,经 `--format ` 到达用户,理由见上一节。`[pack]` 的内建 +模式不需要再添任何新成员。 + +本文档随 `mcpp pack` 实现演进,最新选项以 `mcpp pack --help` 为准。 diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index 45c87e6b8..99da6a361 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -44,7 +44,7 @@ C++23 模块对编译器版本较为敏感,不同版本的 GCC / Clang 在模块 一切命名由两条正交轴构成: -- **toolchain** = `family@version`,family ∈ `gcc | llvm | msvc` ——*用谁编* +- **toolchain** = `family@version`,family ∈ `gcc | llvm | msvc | emsdk | android-ndk` ——*用谁编* - **target** = 三段 triple `arch-os[-env]`(如 `x86_64-linux-musl`、 `x86_64-windows-gnu`、`aarch64-macos`)——*产出给谁* @@ -448,6 +448,102 @@ CRT。 toolset 自带的那份可再分发 CRT(`vcruntime140.dll` / `msvcp140.dll`)可以跟着 产物走 —— 见 `docs/zh/04-mcpp-toml.md` 的 `cxx_runtime = "toolchain-coupled"`。 +## SDK 工具链(`emsdk`、`android-ndk`) + +五种工具链拼法里有两种命名的是一个 **SDK** 而不是一个裸编译器:`emsdk` 与 +`android-ndk`。它们的编译器**就是** clang —— 所以它们不是一个独立的编译器 family, +mcpp 也不假装它们是 —— 而那份归档自带 sysroot、自带 C 库,并且这两者都自带一份 +生成好的 `std` 模块面。本节讲的就是这个差别。 + +### 默认值:该行自己的钉 + +一个目标行命名了它自己的载荷,而那个钉就是默认值。这两者都不需要在 `mcpp.toml` +里写一行: + +```bash +mcpp build --target wasm32-emscripten # 解析到 emsdk@6.0.9 +mcpp build --target aarch64-linux-android # 解析到 android-ndk@30.0.16248370 +``` + +载荷在某个目标第一次需要它时**按需安装**,和一个 gcc 或 llvm 载荷完全一样。 +`mcpp toolchain list` 会在行旁边显示那个钉,而构建会报出是哪份归档回答的: + +``` +Resolved emsdk@6.0.9 → wasm32-emscripten → .../xim-x-emsdk/6.0.9/emscripten/em++ +Resolved android-ndk@30.0.16248370 → aarch64-linux-android → .../prebuilt/linux-x86_64/bin/clang++ +``` + +### 也可以显式声明 + +普通的按目标键照常可用,而点名该行自己的载荷总是被接受: + +```toml +[target.aarch64-linux-android] +toolchain = "android-ndk@30.0.16248370" + +[target.wasm32-emscripten] +toolchain = "emsdk@6.0.9" +``` + +用它把版本跨机器钉住,或者选用比该行约定更新的载荷。**版本是自由的** —— 索引里 +发布过的都能解析 —— 所以一个工程就是用它走在默认值之前或留在它之后。 + +### 不可覆盖的部分及其依据 + +对这两行,那个钉是一个**能力**而不是一个约定:它不是 mcpp 在几个都能服务该目标的 +载荷之间的偏好,而是唯一能服务它的东西。所以载荷的**名字**是固定的,版本是开放的: + +```toml +[target.aarch64-linux-android] +toolchain = "llvm@22.1.8" # 被拒绝 +``` + +``` +error: target 'aarch64-linux-android' cannot be emitted by 'llvm@22.1.8'. + An Android target needs bionic, not just an aarch64 or x86_64 back end: + its headers, its per-API-level stubs and its loader path are inside the + NDK, and no package adds them to another compiler. +``` + +这次拒绝与代码生成无关。一个普通 clang 发 aarch64 ELF 完全没问题;它拿不出来的是 +**体系**。在声明处就说出来,比解析出 llvm 再在构建深处失败要好 —— 而后者正是这道闸 +存在之前发生的事:先是 `'__config' file not found`,然后是 bionic 自己头文件里的 +`Unversioned target triples are not supported!`,两句都没点名那个服务不了这一行的 +工具链。 + +`wasm32-emscripten` 按同一条规则、用它自己的句子拒绝:除了 Emscripten 没有东西发 +WebAssembly。 + +### 属于工程的那一半 + +工具链是 SDK 的;**部署下限**是工程的,而它按平台各有自己的键 —— 见 +[04 — mcpp.toml](04-mcpp-toml.md) §2.7.3: + +```toml +[target.aarch64-linux-android] +min_api_level = 24 # Android +``` + +```toml +[package] +macos_deployment_target = "14.0" # Apple +``` + +一个 NDK 服务一个 API level 的**区间**,所以级别是工程的决定,而点名 +`android-ndk@` 并不钉住其中任何一个。不写的话,mcpp 读 NDK 自己在 +`meta/platforms.json` 里声明的下限。 + +### 产物的运行方式 + +模拟器和真机都不属于工具链这根轴。`mcpp run` 直接执行一个 wasm 模块,因为 +Emscripten 的产物就是一个 `node` 能跑的程序。对于产物在别处运行的目标,`runner` +键是一个 argv 前缀,而那个会话属于一个**包**而不属于引擎: + +```toml +[target.x86_64-linux-android] +runner = ["adb-run"] # 来自 xim:android-platform-tools 的一个程序 +``` + ## 项目级版本锁定 若项目需固定特定版本而不依赖全局默认,可在项目的 `mcpp.toml` 中声明: @@ -713,7 +809,10 @@ Windows 组件(Win10 起),mcpp 从不分发它;而 `vcruntime140.dll` / 模式(`--mode system`、`--mode static`)兑现不了 `toolchain-coupled`,会直接拒绝。 **边界。** 该契约只管 C++ 运行时。静态 **libc** 是另一根轴(`linkage = "static"` -/ `--static`,如 musl 目标),部署下限是第三根轴(`macos_deployment_target`)。 +/ `--static`,如 musl 目标),部署下限是第三根轴 —— Apple 目标用 `[package]` 里的 +`macos_deployment_target`,Android 用 `[target.
]` 下的 `min_api_level`。 +两者喂给 `llvm_triple()` 的同一个参数、占构建指纹里的同一个槽:一个目标要么是 +Apple 要么是 Android,而两者回答的是同一个问题。 另外,`host-coupled` 只承诺 mcpp 不做任何"把 C++ 运行时打进产物"的动作,它不会 去掉链接因其它原因已经携带的工具链 rpath —— 所以在 ELF 上这类产物仍可能优先 找到工具链的库。 diff --git a/docs/zh/21-the-target-triple.md b/docs/zh/21-the-target-triple.md index 9707ca39a..592217bab 100644 --- a/docs/zh/21-the-target-triple.md +++ b/docs/zh/21-the-target-triple.md @@ -32,18 +32,32 @@ C 库。选中 `x86_64-linux-musl` 就是选中 musl-gcc 载荷,选中 | 段 | 内容 | 例 | |---|---|---| -| `arch` | 指令集 | `x86_64`、`aarch64`、`riscv64` | -| `os` | 操作系统,或 `none` | `linux`、`windows`、`macos`、`none` | -| `env` | 见下 —— 它在每个平台上是不同的轴 | `gnu`、`musl`、`msvc`、`elf` | +| `arch` | 指令集 | `x86_64`、`aarch64`、`riscv64`、`wasm32` | +| `os` | 操作系统,或 `none` | `linux`、`windows`、`macos`、`ios`、`emscripten`、`none` | +| `env` | 见下 —— 它在每个平台上是不同的轴 | `gnu`、`musl`、`msvc`、`android`、`elf` | 第三段值得留意,因为它在各处命名的并不是同一类东西: | 平台 | `env` 命名 | 取值 | |---|---|---| -| `linux` | **C 库** | `gnu`(glibc)、`musl` | +| `linux` | **C 库** | `gnu`(glibc)、`musl`、`android`(bionic) | | `windows` | **对象 ABI** | `gnu`(Itanium C++ ABI)、`msvc`(微软的) | | `none` | **对象格式** | `elf` | -| `macos` | 无;该平台不带这一段 | — | +| `ios` | **真机还是模拟器** | `sim`(模拟器)、缺省(真机) | +| `macos`、`emscripten` | 无;该平台不带这一段 | — | + +`sim` 是 Apple 这一侧唯一带段的行,而它命名的既不是 C 库也不是 ABI:模拟器构建 +有自己的 SDK(`iPhoneSimulator.sdk`)、产出自己的对象,并且取 +`-mios-simulator-version-min` 而真机取 `-miphoneos-version-min`。两个目标,所以 +两个身份。Apple 自己把它拼成 OS 段尾部的 `-simulator`,Rust 拼成 +`aarch64-apple-ios-sim`;两种拼法在这里都能解析,并且都规范化为 +`aarch64-ios-sim`。 + +`android` 是一个 **C 库**,所以它落在 `musl` 落的那个位置上,OS 段仍是 `linux`。 +这个位置就是这处建模决定的全部:内核**就是** Linux,所以 ELF、`unix` family、 +`nasm -f elf64` 全都已经是对的;而一个 `os = "android"` 会让这三样默认全错,并 +且要求在每一处站点给出一个新答案。它与 `gnu` 的差别是 bionic、加载器路径和 SDK +—— 而这恰好就是 `env` 这一段存在的意义。 在 Windows 上这一段经常被读错,因为 `gnu` 这个词暗示了一个并不在场的 C 库。 对一份按构建期体系为 `x86_64-windows-gnu` 构建的产物实测: @@ -59,6 +73,27 @@ compiler-rt,C 库是 musl,C++ 运行时是 libc++,平台是 openkal。`gnu` 是 LLVM 词表里「非 MSVC 的那套 ABI」的标签,继承自 MinGW,而 clang 需要这个 拼写来选中正确的内部工具链。mcpp 改不了它。 +### 对象格式是一个轴,不是一处推导 + +一个三元组的二进制格式过去根本不是任何东西:它在每一处需要它的地方从 `os` 重新 +推导一遍。`is_pe()` 问 `os == "windows"`,产物命名再问一遍,打包器问第三遍。答案 +只有两个取值时,这是负担得起的。 + +`wasm32` 是 mcpp 词表里第一个格式不属于那两个的目标,而第三个取值会把那些推导变成 +**在每一处这样的站点上的一次添加** —— 而漏掉的那一处不会报错。它会静默地答 ELF, +因为 ELF 正是这棵树里每一个 `else` 分支所假设的东西。于是格式现在是一个答案: + +| 目标 | 格式 | +|---|---| +| `x86_64-linux-gnu`、`aarch64-linux-android`、`riscv64-none-elf` | ELF | +| `aarch64-macos`、`aarch64-ios` | Mach-O | +| `x86_64-windows-gnu`、`x86_64-windows-msvc` | PE | +| `wasm32-emscripten` | wasm | + +它与「有没有一个操作系统可供链接」**不是**同一个问题。一个裸机 RISC-V 映像是 ELF +且没有 OS;一个 wasm 模块有一层类 OS 的东西(Emscripten 的 POSIX 模拟)而不是 +ELF。把这两个轴并成一个,正是这处改动要消除的那个错误。 + ## 省略第三段 `-` 在每个平台上都是一个完整的目标: @@ -398,6 +433,14 @@ CRT;图供给时是 `musl`。一个目标字符串,两个不同的 C 库 —— | `thumbv8m.base-none-eabi` | preview | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | | `thumbv8m.main-none-eabi` | verified | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | | `thumbv8m.main-none-eabihf` | preview | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | +| `armv7a-none-eabi` | verified | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | +| `armv7a-none-eabihf` | verified | `llvm@22.1.8` | 载荷 | 载荷 | 载荷 | 载荷 | +| `aarch64-linux-android` | preview | `android-ndk@30.0.16248370` | payload | payload | payload | — | +| `x86_64-linux-android` | verified | `android-ndk@30.0.16248370` | payload | payload | payload | — | +| `aarch64-ios` | planned | — | planned | planned | planned | planned | +| `aarch64-ios-sim` | planned | — | planned | planned | planned | planned | +| `x86_64-ios-sim` | planned | — | planned | planned | planned | planned | +| `wasm32-emscripten` | verified | `emsdk@6.0.9` | payload | payload | payload | payload | `载荷` 这里有工具链载荷产出它 · `图` 没有载荷,但依赖可以供给系统 · `系统` 在机器上被找到,不是 mcpp 装的 · `SDK` 平台自己的 · @@ -414,11 +457,27 @@ CRT;图供给时是 `musl`。一个目标字符串,两个不同的 C 库 —— | `x86_64-windows-musl` | 载荷只在 Windows;走图则任意宿主 | 没有 gcc 发得出 PE+musl,而 LLVM 拼不出这个三元组 | | `aarch64-macos` | macOS | SDK 是那台机器的 | | `*-none-elf` | 每一台 | clang 与 lld 按构造就是交叉编译器 | +| `wasm32-emscripten`、`*-linux-android` | 每个宿主 | SDK 自带 sysroot,且上游按宿主发布;一份归档服务每个 guest 架构 | **一个 `—` 讲的是载荷,不是可能性。** `host_can_serve` 回答的是「这里有没有 载荷产出它」,而依赖图可以改为供给系统 —— 这就是 `x86_64-windows-musl` 在 Linux 上显示 `via dependency graph`、并在那里产出真正的 PE32+ 的原因。 +**而两个 Android 行在 Windows 上的 `—` 是**索引**的答案,所以 +`mcpp toolchain list` 在那里仍然会显示它们。** Google 确实发布 Windows NDK,它也 +下载得到;它不包含的是 libc++ 的**模块面**(实测:没有 `std.cppm`、没有 +`std/*.inc`,而另两个宿主各有 110 个),所以 `xim:android-ndk` 不声明 windows 表 —— +一条永远不能服务「模块优先」构建的条目比没有更坏。引擎不把这件事编进来:一个索引 +服务哪些宿主会在没有引擎发布的情况下变化,而把它写成这里的一个常量,正是 wasm 那 +一行自己的历史所展示的会变陈旧的东西。因此 Windows 用户会看到这一行、钉能解析, +而 xim 在任何东西被下载之前以 `no payload for this platform` 拒绝,并点名那个包。 + +**两个 Android 行层级不同,是因为其中一个被运行过。** 一个 x86_64 的 Android 产物 +在平台自己的模拟器上执行得起来,而 `verified` 这个层级断言的正是「做过这件事」。 +真机那一行构建方式完全相同,而从一台 x86_64 宿主没有执行路径:模拟器直接拒绝异构 +guest(`QEMU2 emulator does not support arm64 CPU architecture`),所以它需要一台 +arm64 宿主,或者 qemu-user 那条路。 + ### 而 CI 把每一台都测了 [`ci-target-matrix.yml`](../../.github/workflows/ci-target-matrix.yml) 在全部四台 diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index 912576350..b5dcffa93 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -154,6 +154,81 @@ Target x86_64-windows-gnu → x86_64-w64-windows-gnu (gnu selects the Itanium 没有操作系统时是对象格式 —— 因此记录的是**它是哪一个**, 而不是一个只记录「是否为第一种」的布尔。 +## 这个模型下的 Android、Web 与 iOS + +mcpp 在 2026年9月11日.3 里加出目标行的这三个平台不是同一个问题。决定每一个的是它的 +实现相对一个 C 库该落在哪一侧,而三个答案各不相同。 + +### Android 共用 Linux 的实现,一行都不用改 + +`openkal-linux` 写在 Linux 内核自己的系统调用接口上,不向任何 C 库借用任何东西 —— +这正是它能被放到一个 C 库**底下**的原因。Android 的内核**就是** Linux,给定架构上 +的系统调用 ABI 完全相同,而 `src/sys.h` 按 `__x86_64__` / `__aarch64__` 分支,也就是 +按**架构**而不是按操作系统。它里面没有任何属于 glibc 或 bionic 的东西。 + +所以一个可移植程序不需要新增任何一行。`cfg(os = "linux")` 对一个 Android triple +**为真**,因为 Android 是 `linux` OS 上的一个 `env` 值 —— +[21 — 目标三元组](21-the-target-triple.md) 记着这处建模决定 —— 于是实现由一个 +Linux 消费者本来就会写的那一行选出: + +```toml +[target.'cfg(os = "linux")'.dependencies] +openkal-linux = "0.12.0" +``` + +实测 2026年09月11日,一个只针对 openkal 写的程序 —— 没有 C 库,也没有 `import std`: + +``` +mcpp build --target x86_64-linux-android + kernel-abi openkal (openkal-linux@0.12.0, graph) + -> ELF 64-bit LSB pie, x86-64, interpreter /system/bin/linker64 + +mcpp build --target aarch64-linux-android + -> ELF 64-bit LSB pie, ARM aarch64, 同一个 interpreter +``` + +而那个 x86_64 产物被推到一台 API 24 的模拟器镜像上执行: + +``` +openkal: 1-2-3 exit 0 +``` + +`openkal-linux` 自己也能为两个 Android 目标原样编译,这是两条里较弱的那一条,值得 +分开陈述:前者说的是**实现**构建得起来,后者说的是**它上面的程序**跑得起来。 + +### iOS 会共用 macOS 的实现,而这一条现在还不能声称 + +同样的论证在 Apple 这一侧成立 —— iOS 与 macOS 共用 Darwin 内核,而 `openkal-macos` +是按同样方式按架构分支的 —— 但**论证不是证据**。iPhoneOS 与 iPhoneSimulator 的 SDK +在 Xcode 里且不可再分发,所以 `aarch64-ios` 与 `*-ios-sim` 三行是 `planned`:没有 +东西可以拿来构建,因此也没有东西可以拿来运行。仅凭一个结构性论证就声明支持,是这个 +生态已经付过代价的那种形状 —— 一个在索引里的包不等于一个能构建真实工程的包 —— +所以在 SDK 可达之前,这三行什么都不声称。 + +### Web 需要一份新的实现,而且是另一种形状 + +Emscripten 是三者里**改变模型**而不是扩展表格的那一个。那里没有内核,也没有系统 +调用可发:Emscripten 在一个 JavaScript 宿主之上供给它自己的 C 库。所以给它写的 +openkal 实现不可能按 `openkal-linux` 的方式写 —— 落在一个 C 库底下 —— 而必须落在 +一个 C 库**之上**。规范恰好允许这一点(「一个实现可以建立在一个 C 库之上、之下, +或者不依赖 C 库」),所以这是**新软件**而不是一个共用决定,而它是三者里既没做完也 +没被阻塞的那一个。 + +在它出现之前,`wasm32-emscripten` 走的是普通那条路:一个载荷。`xim:emsdk` 自带 +编译器、sysroot 和一份 libc++ 的模块面,所以一个用 `import std` 的程序今天就能为 +Web 构建并运行,而 openkal 完全不参与 —— 这正是那一行的 `verified` 层级所记录的。 + +### 表 + +| 平台 | 实现 | 状态 | +|---|---|---| +| Linux(glibc、musl) | `openkal-linux` | 参考实现 | +| Android(两个 ABI) | `openkal-linux`,原样 | 构建通过;它上面的程序在模拟器上跑过 | +| macOS | `openkal-macos` | 在 macOS 的系统调用面上 | +| iOS、iOS 模拟器 | `openkal-macos` 会服务它 | 阻塞:SDK 不可再分发 | +| Windows | `openkal-windows` | 在 Win32 与对象管理器上 | +| Web(Emscripten) | 无 | 需要一份写在 C 库**之上**的实现 | + ## 裸机 一个没有操作系统的目标,是同一个模型,只是平台层由固件而非内核供给。 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index e4587085b..2552d352f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -399,6 +399,11 @@ int main() { | `object` | 进**链接**集 | 链接边消费它们 | 资源编译器、`objcopy` 嵌 blob、生成的 `.def`、预编译 `.o` | | `artifact` | 一个新文件 | 它的**输入**是链接产物,所以在链接之后跑 | 签名、打包、size budget | +`artifact` 也是唯一允许写 `${mcpp.stage_dir}` 的 role —— 见下文 +[产出可分发物](#产出可分发物pack_format-与-stage_dir20269111)。另外三个跑在链接之前 +或与链接并行,没有任何已暂存的东西可读,所以 mcpp 会拒绝这个占位符,而不是把它展开成 +一个恰好存在的路径。 + 全程不涉及任何 phase 机制。`object` 与 `artifact` 由 ninja 自己的文件依赖定序 —— 这也是为什么 `artifact` 不会像朴素的「post 构建钩子」那样把自己重复施加一遍。 `source` 与 blocking 的 `check` 则由一条 order-only 边定序:从声明它的那个包的 @@ -470,6 +475,78 @@ mcpp 会播下一个带着该声明的占位文件,使 prepare 期的扫描与 内容一致 —— 与 `[modules].scan_overrides` 同一条「声明 + 验证」的取舍,build 期由 编译器自己的 P1689 输出复核。 +### 产出可分发物:`pack_format` 与 `stage_dir`(2026年9月11日.1+) + +一个 `.msi`、一个 `.deb`、一个 AppImage、一个签过名的 `.app`,都不是那四个 role 的 +惯常活计,而它们全都是 `artifact`:每一个都消费**链接产物**,产出用户去安装的东西。 +引擎为它们加的是一套机制,而不是任何一种格式。 + +`mcpp pack --format ` 把 `` 交给解析后的图去解决,方式与 `--target` +够到一个引擎不必逐个认识的三元组相同。`tar` 与 `dir` 仍然是 `mcpp pack` 自己拥有的 +归档形状;此外的一切都来自某个包。 + +一个提供方有两半,而这两半不许被并成一半: + +```cpp +import mcpp; +#include +#include + +int main() { + // 第一半,无条件。 + mcpp::provides_pack_format("appimage"); + + // 第二半,有条件。 + if (std::string_view(mcpp::pack_format()) != "appimage") return 0; + + const std::string out = std::string(mcpp::out_dir()) + "/app.AppImage"; + mcpp::action a; + a.id = "appimage"; + a.role = "artifact"; + a.arg(tool).arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +``` + +**无条件声明,有条件提交。** 声明是让引擎能回答一个发起请求的那次构建自己回答不了的 +问题:`--format bogus` 要点名**当下确实可用**的那些格式,`--help` 要说「解析后的图 +提供的任何格式」。两者读的都是一次「什么格式都没要」的 pass 收集到的集合。一个只在被 +问到时才声明的成员,对它的作者仍然照常工作 —— 作者永远传的是自己那个格式 —— 而对其他 +所有人,这个集合变成不可知的。对一个谁都没为之提交的格式,mcpp 会拒绝,而不是报告一次 +「什么包都没产出」的成功打包。 + +**`mcpp pack --format ` 会 prepare 两次。** 一条 `artifact` action 是一条 +ninja 边,而那棵暂存树是 mcpp 在链接**之后**产出的,所以这棵树不可能成为构建出它自己 +那一次 pass 的输入。第一次 pass 收集声明,并在任何东西被编译之前拒绝未知的格式;随后 +才是构建与暂存;第二次 pass 设上 `pack_format` 与 `pack_stage_dir`,并构建提供方提交 +的那条边。第二次 pass 里没有任何值是重新推导出来的 —— 三元组与暂存路径都是第一次 +pass 和那次暂存已经回答过的。 + +**暂存树是一个 bundle,不是一个根文件系统。** 它就是 `--mode vendored` 的含义: +`bin/`、`lib/`,可重定位、根在哪儿都行,并且已经过了 strip 策略、调试信息拆分与 +`include`/`exclude`。一个 AppImage、一个 `.app`、一个 `.msi` 要的就是它现在这个样子。 +而要一棵 FHS 树的格式(`.deb`、`.rpm`)自己负责重排布局,因为一个文件该落在哪个目录是 +那个格式的知识,不是引擎的。 + +`mcpp pack --format dir` 把同一棵树写到一个路径上就停下,这是人去查看一个成员将会拿到 +什么的方式。 + +**`mcpp pack` 报告的是这次请求**引入**的那些 artifact action。** 一条无论有没有人 +要格式都在场的 action —— 一个签名 stamp、一次 size budget —— 不是可分发物,点名它会 +是一个看起来像对的错答案。这个判据里没有任何一项是成员的性质:一个只打包一个具名程序、 +从不读暂存树的格式,与一个消费整个闭包的格式被同等识别。对一个谁都没为之提交的格式, +会被点名拒绝。 + +**写了 `${mcpp.stage_dir}` 的 action 会自动获得一条对这棵树的 manifest 的依赖。** +mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是成员,所以它不会跑进任何 +人的安装包里 —— 逐条列出每个已暂存文件的大小与相对路径。这条依赖由引擎添加,因为「用 +了」本身就意味着「依赖」:没有它,这条边只在链接产物变化时才变脏,而一个闭包多出了某个 +依赖的共享库、同时程序自己的字节并没有变的情况,会把上一次的可分发物原地留下,并报告为 +已是最新。 + 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: @@ -479,6 +556,7 @@ mcpp 会播下一个带着该声明的占位文件,使 prepare 期的扫描与 | `${mcpp.bin_dir}` | 产出的二进制所在目录 | | `${mcpp.compile_db}` | `compile_commands.json` 的路径(clang-tidy 的 `-p` 要的就是它) | | `${mcpp.target_file:}` | target `` 构建出的文件 | +| `${mcpp.stage_dir}` *(2026年9月11日.1+)* | `mcpp pack` 暂存出的那棵树,绝对路径。仅 `artifact` role 可用,且仅在 `mcpp pack --format ` 下可用 | 上面的裸 stdout 协议仍是底层基底;`import mcpp;` 是其上的类型化层。 @@ -575,6 +653,13 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 | `MCPP_LANGUAGE_MODULES` *(2026年9月7日.1+)* | -- | 声明它的那个包设了 `[language] modules` 时为 `1`,否则 `0`。**生成**面向消费者声明的规则读它来在模块接口与头文件之间选择,项目因此只需说一次。旧引擎不设这个变量,规则把缺席读作 `0` —— 也就是这个变量存在之前每个消费者的行为 | | `MCPP_PKG_NAME` *(2026年9月7日.1+)* | -- | 这个程序所构建的包的 `[package] name`。规则生成的每个名字都由它推导:消费者导入的模块、访问器所在的命名空间、生成头里的符号。在它存在之前,可用的最接近的答案是 `MCPP_MANIFEST_DIR` 的末段,那是目录名 —— 于是一个叫 `vulkan-saxpy` 的包放在名为 `app` 的目录下会生成 `app.shaders`,而工作区里每一个 `/app/` 都声称拥有同一个模块。旧引擎下缺席,规则把缺席读作「沿用先前的推导」 | | `MCPP_PKG_NAMESPACE` *(2026年9月7日.1+)* | -- | `[package] namespace`。包未声明命名空间时为空。需要产出在索引范围内唯一的名字的规则用这一对而不是单用名字,因为包身份是 `(namespace, name)` | +| `MCPP_PKG_VERSION` *(2026年9月11日.1+)* | `mcpp::package_version()` | `[package] version`。每一种安装包格式都要写版本号;在这个变量之前,项目只能把版本号在成员自己的 options 里再写一遍,而那份副本会与 `[package]` 漂移,且没有任何东西能发现 | +| `MCPP_PKG_DESCRIPTION` *(2026年9月11日.1+)* | `mcpp::package_description()` | `[package] description`。包未声明时为空 | +| `MCPP_PKG_LICENSE` *(2026年9月11日.1+)* | `mcpp::package_license()` | `[package] license` | +| `MCPP_PKG_AUTHORS` *(2026年9月11日.1+)* | `mcpp::package_authors()` | `[package] authors`,以 `;` 连接。不用 `,`:一条 author 的惯例写法是 `Name `,名字里可能带逗号,以逗号连接的列表无法再切回原来的条目 | +| `MCPP_PKG_REPO` *(2026年9月11日.1+)* | `mcpp::package_repo()` | `[package] repo` | +| `MCPP_PACK_FORMAT` *(2026年9月11日.1+)* | `mcpp::pack_format()` | 本程序所处的这次 `mcpp pack` 的 `--format` 取值;任何普通构建下都为空。承载含义的正是这个空值 —— 成员据此为自己的提交加闸,于是 `mcpp build` 拿到的还是它一直以来的那张图 | +| `MCPP_PACK_STAGE_DIR` *(2026年9月11日.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置,绝对路径;本次构建不在打包时为空。读它来判断这次要干的活是什么形状,而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | | `MCPP_DEVICE_SOURCES` *(2026年9月5日.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件(`.cu`、`.hip`...),相对包根,一行一个;没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄:构建未覆盖的 `{ glob, accel }` 条目贡献为空,因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出/暂存目录 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根(= CWD) | @@ -668,7 +753,16 @@ shim,而可用的那份就在项目自己的环境里,根本不在 `PATH` 上。 什么都不会坏;只是这个名字声称了一个该包并不具有的来源。项目之外的规则自选前缀。 -**工具不是规则。** 规则说明一个编译单元如何被 mcpp 并不驱动的编译器编译:它提交一条 +**工具不是规则,而可分发物两者都不是。** 三类活计,成员的前缀说明它回答三个问题中的 +哪一个: + +| 前缀 | 回答的问题 | 编译编译单元 | 在构建程序里执行 | +|---|---|---|---| +| `rules-*` | 这个编译单元如何被编译 | 是 | 否 | +| `tools-*` | 构建程序自己需要做什么 | 否 | 是 | +| `dist-*` *(2026年9月11日.1+)* | 链接之后出来的是什么,以及用户以什么形态安装它 | 否 | 否 | + +规则说明一个编译单元如何被 mcpp 并不驱动的编译器编译:它提交一条 action,由引擎调度。工具说明的是构建程序需要、而没有任何编译器执行的事,并在构建程序 运行时当场做掉。`mcpp.tools.embed`(feature `tools-embed`,mcpp 2026年9月5日.4+)是第一个: 它把数据文件写成程序编译进去的头文件(字节数组或 32 位字数组),内容未变时不重写文件, diff --git a/docs/zh/31-authoring-a-rule-package.md b/docs/zh/31-authoring-a-rule-package.md index fabeaa331..5b0060b73 100644 --- a/docs/zh/31-authoring-a-rule-package.md +++ b/docs/zh/31-authoring-a-rule-package.md @@ -269,6 +269,50 @@ mcpp::floor("cuda.driver", runtime_needs); 一个 option 暴露出来,好让用别的键声明这条边的消费者能提供它;并且在找不到时以 一条点名「找的是什么」的消息拒绝,而不是拿一个空路径去执行命令。 +## 编写一个分发成员(mcpp 2026年9月11日.1+) + +一个 `dist-*` 成员既不是规则也不是工具。它不编译编译单元,也不在构建程序运行期间把活 +干完:它消费**链接产物**,产出用户去安装的东西 —— 一个 `.msi`、一个 `.deb`、一个 +AppImage、一个签过名的 `.app`。机制是一条 `artifact` action 加上 +`mcpp pack --format `,见 +[30 —— 产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111)。 + +本章其余内容原样适用。有六条在这里绑得更紧,每一条都是这个类别会犯、而规则不会犯的 +错。 + +**无条件声明,有条件提交。** `provides_pack_format` 是引擎用来回答「这张图提供哪些 +格式」的东西,而回答发生在一次「什么格式都没要」的构建上。一个只在被问到时才声明的 +成员,对它的作者照常工作 —— 作者永远传自己那个格式 —— 而对其他所有人,这个集合变成 +不可知的。 + +**同时提供 plan 与 submit。** 可分发物是交到用户手上之前的最后一环,因此它是整个构建 +里最可能需要项目自己改一笔的部分:换一个压缩等级、多带一个文件、加第二个签名。 +`generate_all(opt)` 恰好等于 `submit(plan_all(opt))`,正是让这样一笔改动不至于变成把 +成员重新实现一遍的东西。 + +**点名输入,不要去 harvest 一个目录。** 一个解析成空的路径是无声的,而一个缺失的具名 +输入是错误。实测:一条 WiX action 绑定一个目录并 harvest 它,当那个路径在 Windows 上 +解析成空时,产出了一个**有效的、空的、52 KB 的安装包,并且没有任何诊断**。 +`${mcpp.target_file:}` 就是这个机制 —— 构建程序既不知道三元组也不知道指纹,而一个 +不存在的 target 名会被拒绝,而不是展开成一个空路径。 + +**在成功路径上,为自己的输出断言一条下界。** 只要成员能判断自己的结果是空的或不合理 +的,它就必须经 `mcpp::warning` 说出来,因为成功构建的 stderr 会被丢弃。这与上一段实测到 +的是同一个失败,只是被拦在了后一层。 + +**在工具会被查找的那个位置声明它。** `xpkg_dir` 从 `MCPP_XPKG_*_DIR` 回答,而这些是 +mcpp 为**正在被构建的那个包**设置的。一个依赖的声明会把载荷装上,却不会让它对消费者的 +构建程序可见,所以运行载荷工具的成员要自己声明它 —— 并且在查找返回空时把这件事说出来, +而不是指向一份读者并不拥有的 manifest。 + +**构建不许碰网络,而被包起来的工具可能会碰。** 在 `appimagetool` 1.9.1 上实测:除非用 +`--runtime-file` 指定一份本地副本,它每次被调用都会从一个 GitHub release 下载它的 +type-2 runtime 存根。包装这类工具的成员必须从自己声明的载荷里把这个文件供上。安装期 +是下载合法的时候;构建期不是,而一个会去取东西的构建既不可复现也不能离线用。 + +**一个 `(name, version)` 只指一份载荷。** 包装签名或打包工具的成员继承了那个工具的兼容 +面,所以与被包装的工具同步版本是正当的,而且陈述了一件真事。 + ## 当前边界 - 规则包自己 `[features] default` 里的规则 feature 不隐含 `host-module`。 diff --git a/docs/zh/README.md b/docs/zh/README.md index f3b7dc060..f44cb1d43 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -121,6 +121,7 @@ | `[build] accel`、`[package] accelerators`、`device_extensions` | [42](42-heterogeneous-builds.md) | `[hooks]` | [09](09-commands-by-scenario.md) | | `[package] platforms`、`[build] cache` | [04](04-mcpp-toml.md) | `[targets.]`、`[profile.]` | [04](04-mcpp-toml.md) | | `runner`、`[target..runners]` | [41](41-devices.md) | `rule_module` | [31](31-authoring-a-rule-package.md) | +| `min_api_level` | [04](04-mcpp-toml.md) | `macos_deployment_target` | [04](04-mcpp-toml.md) | **命令** diff --git a/examples/09-heterogeneous/multi-backend/mcpp.toml b/examples/09-heterogeneous/multi-backend/mcpp.toml index 12fadb91b..b5cdccd9c 100644 --- a/examples/09-heterogeneous/multi-backend/mcpp.toml +++ b/examples/09-heterogeneous/multi-backend/mcpp.toml @@ -104,7 +104,7 @@ cuda-driver = "2026.09.05" # is a payload and the hardware ones are the host's. [target.'cfg(accelerator = "vulkan")'.dependencies.compat] vulkan = "1.4.357.0" -vulkan-runtime = "2026.09.10" +vulkan-runtime = "2026.09.11" [build] # The device sources carry the accel they are for. A CONSTRAINED glob gates diff --git a/examples/09-heterogeneous/sycl/app/mcpp.toml b/examples/09-heterogeneous/sycl/app/mcpp.toml index 97ac1963f..325fe272d 100644 --- a/examples/09-heterogeneous/sycl/app/mcpp.toml +++ b/examples/09-heterogeneous/sycl/app/mcpp.toml @@ -37,7 +37,7 @@ plugins = { version = "0.5.2", features = ["rules-sycl", "tools-island"], host-m # message (#596). The farm now mirrors what the driver sentinel publishes # rather than naming a file. [dependencies.compat] -sycl-runtime = "2026.09.10" +sycl-runtime = "2026.09.11" # NO [xlings.workspace]. `mcpp.rules.sycl` declares all five payloads this lane # needs, each closing one hole the host would otherwise fill: `dpcpp` (the diff --git a/examples/09-heterogeneous/vulkan/app/mcpp.toml b/examples/09-heterogeneous/vulkan/app/mcpp.toml index 6e314cdf1..627ce12c4 100644 --- a/examples/09-heterogeneous/vulkan/app/mcpp.toml +++ b/examples/09-heterogeneous/vulkan/app/mcpp.toml @@ -22,7 +22,7 @@ plugins = { version = "0.5.2", features = ["rules-spirv"], host-module = true } # below is a payload and the hardware ones are the host's. [dependencies.compat] vulkan = "1.4.357.0" -vulkan-runtime = "2026.09.10" +vulkan-runtime = "2026.09.11" # ONE payload here, and the shader compiler is not it: `mcpp.rules.spirv` # declares `xim:glslang` for itself. What stays is a DEVICE, and the boundary is diff --git a/examples/10-graphics/offscreen/mcpp.toml b/examples/10-graphics/offscreen/mcpp.toml index febc6e541..46468dc11 100644 --- a/examples/10-graphics/offscreen/mcpp.toml +++ b/examples/10-graphics/offscreen/mcpp.toml @@ -28,14 +28,23 @@ plugins = { version = "0.5.2", features = ["rules-spirv"], host-module = true } # seam work: the CPU leg below is selected by the accelerator, and only the # packages are unconditional. [dependencies.compat] -vulkan = "1.4.357.1" +vulkan = "1.4.357.2" -# The adapter that makes the host's own ICDs reachable from a binary running -# under mcpp's private loader. An OS predicate, which IS allowed, and a Linux -# concern by construction: macOS resolves through dyld and Windows through the -# PE loader, and neither has a private loader to work around. -[target.'cfg(linux)'.dependencies.compat] -vulkan-runtime = "2026.09.10" +# THE ADAPTER IS NOT NAMED HERE, AND THAT IS THE POINT. +# +# `compat.vulkan` declares `compat.vulkan-runtime` itself on Linux -- the +# adapter that makes the host's own ICDs reachable from a binary running under +# mcpp's private loader, which macOS and Windows do not need because dyld and +# the PE loader have no private-loader problem to work around. +# +# Naming it here as well pinned one version in two places, in two repositories, +# with nothing enforcing that they agree. They drifted three times in one day: +# the loader package moved its pin, this file moved its own, and an installed +# copy of `compat.vulkan` still recorded the previous one -- each time the build +# stopped with `irreconcilable versions`, and each time the fix was to edit the +# other place. A dependency that another dependency already declares is a +# second copy of a decision; the version it needs is the one that package says +# it needs. # A Vulkan device that needs no GPU, so this example runs on a machine that has # none. It is a DEVICE and therefore a payload; the drivers a real GPU needs are diff --git a/examples/13-platform-targets/README.md b/examples/13-platform-targets/README.md new file mode 100644 index 000000000..89bd70459 --- /dev/null +++ b/examples/13-platform-targets/README.md @@ -0,0 +1,124 @@ +# 13 — platform targets + +一份源码,三个平台。这里没有任何平台感知的东西:没有 `cfg`、没有预处理分支、 +没有按目标分开的源文件。在 Linux 二进制、WebAssembly 模块和 Android 产物之间 +变的只有命令行上的 `--target`。 + +```bash +cd 13-platform-targets +mcpp build && mcpp run # 宿主 +``` + +## Web + +```bash +mcpp run --target wasm32-emscripten +``` + +实测(linux-x86_64,`xim:emsdk` 6.0.9): + +``` +bin/platform-targets 65389 bytes the JavaScript +bin/platform-targets.wasm 447183 bytes the module +node bin/platform-targets -> 1-2-3 +``` + +`mcpp run` 会用 `node` 跑它,所以不需要额外的一步。工程侧**一个新词汇都不需要**: +`wasm32-emscripten` 这一行自己命名了它的载荷(`emsdk@6.0.9`),载荷自带 sysroot, +而 Emscripten 自己就发布一份 libc++ 的模块面。 + +## Android + +```bash +mcpp build --target x86_64-linux-android # 模拟器 +mcpp build --target aarch64-linux-android # 真机 +``` + +实测(同一台机器,`xim:android-ndk` 30.0.16248370): + +``` +aarch64-linux-android -> ELF 64-bit LSB pie, ARM aarch64, + interpreter /system/bin/linker64 +x86_64-linux-android -> ELF 64-bit LSB pie, x86-64, 同一个 interpreter +``` + +**一个钉服务两行。** NDK 不命名架构,`--target` 才命名 —— 所以 +`[target.
] toolchain` 不需要写,而两行共用 +`android-ndk@30.0.16248370`。 + +跑起来(x86_64 键在平台自己的模拟器上,API 24 镜像 + KVM): + +```bash +adb push target/x86_64-linux-android/*/bin/platform-targets /data/local/tmp/ +adb shell /data/local/tmp/platform-targets +# -> 1-2-3 +``` + +加载时会有一句告警,它**不是**缺陷:`unsupported flags DT_FLAGS_1=0x8000001`。 +API 24 的 bionic 加载器不认识 lld 设置的 `DF_1_PIE` 位,于是告警一句,然后照常 +把程序加载起来。 + +### API level 在 `mcpp.toml` 里,不在 triple 里 + +`mcpp.toml` 声明的是: + +```toml +[target.aarch64-linux-android] +min_api_level = 24 +``` + +规范 triple 保持 `aarch64-linux-android` —— 它命名输出目录、`cfg(env = ...)` +和 ABI tag。级别只进**编译器看到的** triple(`aarch64-unknown-linux-android24`) +和**构建指纹**:级别决定哪些 bionic 符号可见,所以两个级别是两个 ABI,绝不可共用 +一个构建目录。 + +这个键是**可选的**。不写的话,mcpp 读 NDK 自己在 `meta/platforms.json` 里声明的 +下限(r30 是 21)。写在这里是因为一个要发布到某个最低版本的工程应该自己说出来, +而不是继承载荷的下限恰好是多少。 + +`[package] macos_deployment_target` 是 Apple 目标上的同一根轴;一个目标要么是 +Apple 要么是 Android,所以两者在指纹里共用一个槽。 + +## 这一行不能被覆盖 + +Android 和 wasm 的钉是**能力**而不是约定: + +```bash +mcpp build --target aarch64-linux-android # [target....] toolchain = "llvm@22.1.8" +# error: target 'aarch64-linux-android' cannot be emitted by 'llvm@22.1.8'. +# An Android target needs bionic, not just an aarch64 or x86_64 back end: +# its headers, its per-API-level stubs and its loader path are inside the +# NDK, and no package adds them to another compiler. +``` + +一个普通 clang 发 aarch64 ELF 完全没问题 —— 它拿不出来的是**体系**。说出来比 +解析出 llvm 再在它内部失败要好。 + +## iOS + +`aarch64-ios`、`aarch64-ios-sim`、`x86_64-ios-sim` 三行在词汇里,都是 `planned`: + +```bash +mcpp build --target aarch64-ios-sim +# error: target 'aarch64-ios-sim' is registered but not yet supported (planned) +# — no toolchain is published for it yet. +``` + +阻塞项是**许可**而不是载荷:NDK 是 Apache-2.0、Emscripten 是 MIT,而 iPhoneOS 与 +iPhoneSimulator 的 SDK 在 Xcode 里,两者都不可再分发。这三行今天买到的是一句 +点名那一行的 `tier-planned`,而不是一句假的 `unknown target`。 + +模拟器是**一个目标**而不是一个 runner:它有自己的 SDK、产出自己的对象,取 +`-mios-simulator-version-min` 而真机取 `-miphoneos-version-min`。所以它有自己的 +行,而不是折进设备那一行。 + +## 支持矩阵 + +`docs/21-the-target-triple.md` 的表是完整的那一份;这里只列这个例子碰到的行: + +| target | tier | pin | 运行过? | +|---|---|---|---| +| `wasm32-emscripten` | verified | `emsdk@6.0.9` | 是,`node` | +| `x86_64-linux-android` | verified | `android-ndk@30.0.16248370` | 是,平台模拟器 | +| `aarch64-linux-android` | preview | `android-ndk@30.0.16248370` | 否 —— 从 x86_64 宿主没有执行路径 | +| `aarch64-ios` / `*-ios-sim` | planned | — | 否 | diff --git a/examples/13-platform-targets/mcpp.toml b/examples/13-platform-targets/mcpp.toml new file mode 100644 index 000000000..ab53ffd92 --- /dev/null +++ b/examples/13-platform-targets/mcpp.toml @@ -0,0 +1,21 @@ +[package] +name = "platform-targets" +version = "0.1.0" + +# ANDROID'S API LEVEL IS A PROJECT DECISION, NOT A TOOLCHAIN PROPERTY, which +# is why it lives here and not in the triple. One NDK serves a range of levels, +# and the level selects which bionic symbols exist -- so it is an ABI axis and +# it enters the build fingerprint: two levels are two build directories. +# +# It is OPTIONAL. Left out, mcpp reads the floor the NDK itself declares in +# `meta/platforms.json` (21 for r30). It is stated here because a project that +# ships to a minimum should say so rather than inherit whatever the payload's +# floor happens to be. +# +# `macos_deployment_target` in `[package]` is the same axis for Apple targets; +# a target is one or the other, so they share one slot in the fingerprint. +[target.aarch64-linux-android] +min_api_level = 24 + +[target.x86_64-linux-android] +min_api_level = 24 diff --git a/examples/13-platform-targets/src/main.cpp b/examples/13-platform-targets/src/main.cpp new file mode 100644 index 000000000..63c527432 --- /dev/null +++ b/examples/13-platform-targets/src/main.cpp @@ -0,0 +1,16 @@ +// One source, three platforms. Nothing here is platform-aware: no `cfg`, no +// preprocessor branch, no per-target source. The only thing that changes +// between a Linux binary, a WebAssembly module and an Android artifact is the +// `--target` on the command line. +// +// `import std` is the point. A target whose toolchain cannot compile a module +// interface unit would be worse than its absence in a module-first build tool, +// so this deliberately exercises the standard library module rather than a +// header, on every target the README lists. +import std; + +int main() { + std::vector v{3, 1, 2}; + std::ranges::sort(v); + std::print("{}-{}-{}\n", v[0], v[1], v[2]); +} diff --git a/mcpp.toml b/mcpp.toml index 2dfc8efd0..90d13140c 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.10.2" +version = "2026.9.11.3" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index d0557d39b..5dc9dc742 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -135,6 +135,21 @@ enum class Slot : std::size_t { // vendor knowledge in the package that has it and out of the engine. Facts, Floors, + // A DISTRIBUTION FORMAT THIS PACKAGE PROVIDES (`mcpp:pack-format=`). + // + // `mcpp pack --format ` resolves `` through the graph the same + // way `--target` reaches a triple: the engine holds the DISPATCH and no + // format. The value is a bare name and means nothing to this file, which is + // what keeps dpkg's control fields, WiX's schema and Apple's notarisation + // out of an engine whose release would otherwise be coupled to theirs. + // + // COLLECTED FROM A BUILD THAT ASKED FOR NOTHING, which is why it is a slot + // and not a side effect of the request. `mcpp pack --format bogus` names + // what is available and `--help` says "plus any format the resolved graph + // provides"; both read this set on a pass where `MCPP_PACK_FORMAT` is + // empty. See `mcpp::provides_pack_format` for the author-facing half of the + // same rule -- declare unconditionally, submit conditionally. + PackFormats, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -217,7 +232,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -336,6 +351,20 @@ inline constexpr std::array kTable{{ // Slot::Facts for the shape of each value. {"fact", "fact", Slot::Facts, Scope::Claim, Transform::Verbatim, false, "", "", 7}, {"floor", "floor", Slot::Floors, Scope::Claim, Transform::Verbatim, false, "", "", 7}, + // `tag` IS NON-EMPTY FOR THE REASON `warning`'S IS, AND IT MATTERS MORE + // HERE. A build program's result is cached and a hit does not re-run it, so + // a declaration that lived only on the run path would be present on the + // first build of a project and absent on every later one -- and the pass + // that reads it is `mcpp pack`, which is never the first build. The set + // would then be empty exactly when a user asks for a format, and the + // refusal would name nothing. + // + // kCacheEpoch is NOT bumped. An entry written before this row carries no + // `d pack-format` line, and the program that wrote it could not emit one, + // so replaying it yields what that program said. An older engine reading a + // newer entry already discards the whole record through the unknown-tag + // path. + {"pack-format", "pack-format", Slot::PackFormats, Scope::Claim, Transform::Verbatim, false, "", "", 9}, }}; // ── Collected output of one run ──────────────────────────────────────────── @@ -813,6 +842,13 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { m.runtimeConfig.requirements.push_back(std::move(req)); } + // A NAME, CARRIED AND NOT INTERPRETED. The engine compares it against + // `--format` and hands the request to whoever claimed it; nothing here + // parses it, because a distribution format has less claim to a name in the + // engine than a language does, and Slang is already supported without one. + for (auto const& f : d.at(Slot::PackFormats)) + bc.packFormats.push_back(f); + // Build-graph nodes. Decoded here rather than at parse time so the cache // stores the payload verbatim and a replay is byte-identical to a run. for (auto const& payload : d.at(Slot::Actions)) { diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index 7301dcfa7..501374f12 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -66,7 +66,12 @@ export namespace mcpp::build::program_protocol { // `--exclude-libs`) had no way out. Same cost as v5's: a package calling // `mcpp::link_flag()` fails on an older engine at the build.mcpp COMPILE, // because that engine's bundled module has no such function. -inline constexpr int kProtocolVersion = 8; +// v9: adds `pack-format` -- the outlet by which a package says which +// distribution format it provides, so `mcpp pack --format ` can dispatch +// to it. Same cost as v5's: a package calling `mcpp::provides_pack_format()` +// fails on an older engine at the build.mcpp COMPILE, because that engine's +// bundled module has no such function. +inline constexpr int kProtocolVersion = 9; // ── Cache-format epoch ───────────────────────────────────────────────────── // diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 9e567f5be..2a5bb8672 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -2501,6 +2501,23 @@ std::expected parse_string(std::string_view content, e.sysrootDeclared = true; } + // `min_api_level` — the oldest OS release the artifact must run + // on, for a target whose compiler takes it inside the triple. + // Refused rather than coerced when it is not a positive integer: + // a level is a number, and a string that looks like one would be + // a second spelling of the same key. + if (auto it = body.find("min_api_level"); it != body.end()) { + auto n = it->second.is_int() + ? std::optional(it->second.as_int()) + : std::nullopt; + if (!n || *n <= 0) { + return std::unexpected(error(origin, std::format( + "[target.{}].min_api_level must be a positive integer, " + "e.g. min_api_level = 24", triple))); + } + e.minApiLevel = static_cast(*n); + } + // `runner` — the argv template `mcpp run` uses for a target whose // artifact cannot execute here. An ARRAY, so it is neither a // scalar (the unknown-key sweep below skips it by type) nor part diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b8b3718fa..8ea217616 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -385,6 +385,23 @@ struct BuildAction { // two are matched against each other. std::string packageName; Role role = Role::Source; + // Set by the engine, never by the build program: this action's command or + // inputs named `${mcpp.stage_dir}`. + // + // ITS ONE JOB IS THE IMPLICIT DEPENDENCY. An action that names the staged + // tree gains an edge to that tree's manifest, so it is dirty when the + // staged SET changes and not only when a link output does. The dependency + // is implied by the use, so a member author cannot forget it. + // + // IT IS NOT HOW `mcpp pack --format ` DECIDES WHICH ACTION IS THE + // DISTRIBUTABLE, and briefly was. Not every format consumes the closure: an + // `.msi` built from ONE NAMED PROGRAM takes `${mcpp.target_file:}` + // and never looks at the tree -- which is the shape the guidance + // recommends, after a bind path that resolved to nothing produced a valid, + // empty, 52 KB installer. So the member that followed the guidance was the + // member that check refused. The dispatch asks instead which artifact + // actions the REQUEST INTRODUCED; see mcpp.pack.pipeline. + bool consumesStageDir = false; std::vector inputs; // absolute or package-relative std::vector outputs; // ditto; declared, see INV-D // Object only: which link units receive the outputs. Empty = every LINKED @@ -693,6 +710,17 @@ struct BuildConfig : BuildInputs { // (`mcpp:action=`). Empty for every package that does not use one, so an // ordinary build is untouched. std::vector actions; + // Distribution formats this package's build program declared it provides + // (`mcpp:pack-format=`). Empty for every package that ships no such member, + // so an ordinary build is untouched. + // + // THE ENGINE HOLDS THE DISPATCH AND NOT THE FORMAT. `mcpp pack --format + // ` looks the name up in the union of these lists, exactly as + // `--target` reaches a triple the engine did not have to know + // individually. `.deb`'s control fields, WiX's schema and Apple's + // notarisation each couple a release to a release mcpp does not control, + // and a name here is the whole of what the engine learns. + std::vector packFormats; bool staticStdlib = true; // #336 — the C++ runtime DISTRIBUTION contract: what the artifact promises // about the machine that runs it ("self-contained" | "toolchain-coupled" | @@ -1064,6 +1092,30 @@ struct TargetEntry { // Two plain members carry the same information and instantiate nothing. std::string sysroot; bool sysrootDeclared = false; + // THE OLDEST OS RELEASE THIS ARTIFACT MUST RUN ON, for a target whose + // compiler takes it as part of the triple. + // + // `min_api_level = 24` under `[target.aarch64-linux-android]`. Android's + // own term is "API level" -- the NDK's CMake toolchain documents + // `ANDROID_PLATFORM` as "the minimum API level supported by the + // application or library" -- and the quantity here is that minimum. + // + // A PROJECT DECISION AND NOT A TOOLCHAIN PROPERTY, which is why it is a + // manifest key. One NDK serves a RANGE of levels: naming + // `android-ndk@30.0.16248370` does not pin API 24, so the level cannot be + // read off the toolchain. + // + // AND NOT PART OF THE CANONICAL TRIPLE, which is the other half. mcpp + // keeps its own target vocabulary and maps it to a compiler target, and + // this is the same shape `macos_deployment_target` already has: + // + // canonical aarch64-linux-android identity: output dir, cfg(), ABI tag + // effective aarch64-unknown-linux-android24 what clang is given + // fingerprint carries the level so 21 and 24 are two directories + // + // Zero means unset, which is legal and means the NDK's own default -- what + // `clang -target aarch64-linux-android` normalises to. + int minApiLevel = 0; // NO per-role field here. There used to be a `cxxRuntimeTests`, and it was // parsed nowhere and applied nowhere — a configuration key that looked // available and did nothing (#418). The per-target channel carries the diff --git a/modules/toolchain-model/src/linkmodel.cppm b/modules/toolchain-model/src/linkmodel.cppm index 44eec3137..440d2f282 100644 --- a/modules/toolchain-model/src/linkmodel.cppm +++ b/modules/toolchain-model/src/linkmodel.cppm @@ -372,6 +372,28 @@ ToolchainLinkModel resolve_link_model(const Toolchain& tc) { // cross-compile resolves by what it builds FOR. if (is_msvc_target(tc) || is_mingw_target(tc)) return lm; + // AND A TARGET WHOSE TOOLCHAIN SHIPS ITS OWN SYSROOT, for the same reason + // one sentence further up: nothing here describes its C library. + // + // THIS IS THE MODEL AND NOT A CHANNEL, and that distinction is the whole + // reason the gate belongs here. The C-runtime group reaches the link line + // through TWO of them -- `link_toolchain_flags` and `payload_ld`, both + // rendering `lm.link_flags()` -- and the comment at the second one records + // that a reader who fixed only the first "saw the identical error and could + // reasonably conclude the fix had not worked". Measured here too, on + // `--target wasm32-emscripten` with the compile side already correct: + // + // wasm-ld: error: unknown argument: + // --dynamic-linker=/lib64/ld-linux-x86-64.so.2 + // wasm-ld: error: unknown file type: + // /lib64/libatomic.so + // + // This host's loader and this host's libatomic, handed to a WebAssembly + // linker. `CLibMode::None` is what both channels then render, because they + // render the model. + if (auto tt = triple::parse(tc.targetTriple); tt && tt->has_own_sysroot()) + return lm; + // The compiler's OWN runtime lives beside it, not in the C library: // libgcc_s.so.1 for GCC. A produced binary links it whether or not the // build ever mentions it, so its directory has to be on the artifact's diff --git a/modules/toolchain-model/src/model.cppm b/modules/toolchain-model/src/model.cppm index 8229fa7db..448357758 100644 --- a/modules/toolchain-model/src/model.cppm +++ b/modules/toolchain-model/src/model.cppm @@ -497,10 +497,15 @@ std::vector graph_runtime_compile_flags(const Toolchain& tc) { // defect was invisible until a second architecture was built. if (t->arch == "aarch64") out.emplace_back("--rtlib=compiler-rt"); if (t->is_pe()) out.emplace_back("-fdwarf-exceptions"); - if (t->is_pe() || t->os == "macos") out.emplace_back("-femulated-tls"); + // OBJECT FORMAT, NOT OS: `is_mach_o()` covers iOS along with macOS, which + // `os == "macos"` used to miss. Both need the emulated-TLS model for the + // same reason PE does -- `_tlv_bootstrap` is loader-bootstrapped there + // exactly as `_tls_index` is on PE. + if (t->is_pe() || t->is_mach_o()) out.emplace_back("-femulated-tls"); // MACH-O ONLY, AND THE REASON IS THAT WEAK-DEF IS A RUN-TIME MECHANISM - // THERE. See the note on this function for the measurement. - if (t->os == "macos") { + // THERE. See the note on this function for the measurement. `is_mach_o()` + // rather than `os == "macos"`: the mechanism is ld64's, which iOS shares. + if (t->is_mach_o()) { out.emplace_back("-fvisibility=hidden"); out.emplace_back("-fvisibility-inlines-hidden"); } @@ -524,8 +529,11 @@ bool target_supports_full_static(std::string_view targetTriple, bool hostCapabil // false is what keeps the two mechanisms from both emitting the flag. if (t->is_pe()) return false; - // macOS cannot fully static-link: libSystem must stay dynamic. - if (t->os == "macos") return false; + // Mach-O cannot fully static-link: libSystem (macOS) / the Apple + // equivalent (iOS) must stay dynamic. `is_mach_o()`, paired with `is_pe()` + // above, so this reads as "for each object format" rather than leaving + // iOS to fall through to the `linux` line below by accident. + if (t->is_mach_o()) return false; // Linux ELF — glibc or musl, native or cross. This is the line that was // previously gated on the HOST being Linux. diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 2a605a65b..b754beb29 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -29,6 +29,37 @@ import mcpp.platform; export namespace mcpp::toolchain::triple { +// WHAT A TARGET PRODUCES, AS ONE ANSWER RATHER THAN A DERIVATION AT EACH SITE. +// +// The binary format used not to be anything: it was re-derived from `os` +// wherever it was needed -- `is_pe()` asked `os == "windows"`, artifact naming +// asked again, the packer asked a third time -- and that is affordable only +// while the answer has two values. A THIRD produces an addition at every such +// site, and a site that was missed does not fail: it silently answers "ELF", +// because ELF is what every `else` branch in the tree assumes. +// +// That is why this exists before wasm needs it rather than after. `wasm32` is +// the first target in mcpp's vocabulary whose object format is neither of the +// two the tree was written around, and #597 is a target-model change for +// exactly this reason -- not because a table row is hard. +// +// IT IS NOT THE SAME QUESTION AS `is_freestanding()`, and merging them would be +// the mistake this replaces. "Which container do objects come in" and "is there +// an operating system to link against" are different axes: a bare-metal +// RISC-V image is ELF with no OS, and a wasm module has an OS-like layer +// (Emscripten's POSIX emulation) and is not ELF. +enum class ObjectFormat { Elf, MachO, Pe, Wasm }; + +std::string_view to_string(ObjectFormat f) { + switch (f) { + case ObjectFormat::Elf: return "ELF"; + case ObjectFormat::MachO: return "Mach-O"; + case ObjectFormat::Pe: return "PE"; + case ObjectFormat::Wasm: return "wasm"; + } + return "ELF"; +} + struct Triple { std::string arch; // "x86_64" | "aarch64" | "riscv64" | ... (GNU spelling) std::string os; // "linux" | "macos" | "windows" @@ -99,7 +130,25 @@ struct Triple { // LDBL_DIG ('33 == 18') // // 33 is aarch64's binary128; 18 is x87. Two machines in one command line. - std::string llvm_triple(std::string_view macosVersion = {}) const { + // THE EFFECTIVE TRIPLE, WHICH IS NOT THE CANONICAL ONE. + // + // `str()` is mcpp's vocabulary and is the identity: the output directory, + // `cfg()`, the packed ABI tag and the fingerprint all derive from it. This + // is the spelling a COMPILER takes, and the two are deliberately different + // -- `aarch64-macos` against `arm64-apple-macos14.0`. + // + // `minPlatformVersion` IS THE PROJECT'S STATEMENT, passed in rather than + // stored, because it is a manifest value and this function must stay pure + // of the manifest. Two platforms fuse it into the triple and each names it + // in its own words: + // + // macOS the deployment target `[build] macos_deployment_target` + // Android the minimum API level `[target.] min_api_level` + // + // Keeping it one parameter rather than two is the point: both answer "the + // oldest OS release this artefact must run on", and a second parameter + // would let a caller supply one platform's answer for the other's. + std::string llvm_triple(std::string_view minPlatformVersion = {}) const { if (empty()) return {}; if (os == "macos") { // Apple spells the 64-bit ARM architecture `arm64`, and the OS @@ -108,15 +157,68 @@ struct Triple { // decision belonging to the project rather than to the compiler. const std::string a = (arch == "aarch64") ? "arm64" : arch; std::string t = a + "-apple-macos"; - t += macosVersion.empty() ? std::string("14.0") - : std::string(macosVersion); + t += minPlatformVersion.empty() ? std::string("14.0") + : std::string(minPlatformVersion); return t; } if (os == "windows") { if (is_msvc_env()) return arch + "-pc-windows-msvc"; return arch + "-w64-windows-gnu"; } - if (os == "linux") return arch + "-unknown-linux-" + (env.empty() ? "gnu" : env); + // APPLE'S OTHER OS. Same `arm64` spelling and the same vendor segment; + // what differs is the SDK and the deployment-target flag. + // + // NO VERSION IS BAKED IN, unlike the macOS branch above, and that is a + // decision rather than an omission. `-miphoneos-version-min` belongs to + // the layer that also owns the SDK path and the `.app` bundle -- a + // distribution plugin -- and a default written here would be a second + // place that answers it. clang picks its own when nothing says. + if (os == "ios") { + const std::string a = (arch == "aarch64") ? "arm64" : arch; + // AND THE SIMULATOR IS A DIFFERENT EFFECTIVE TRIPLE, WHICH IS WHY + // IT IS A DIFFERENT ROW. Apple spells it with a trailing + // `-simulator` on the OS segment; mcpp carries it as `env = "sim"` + // in three fields, which is Rust's `aarch64-apple-ios-sim` modulo + // the vendor elision this whole table already does. The SDK, the + // object and the `-mios-simulator-version-min` flag all differ from + // the device's, so folding the two into one identity would be the + // mistake `x86_64-windows-musl` was added to undo. + if (env == "sim") return a + "-apple-ios-simulator"; + return a + "-apple-ios"; + } + // ANDROID IS LINUX, AND THE ENV SEGMENT IS WHERE IT SAYS SO -- with the + // API level fused onto it when the project stated one. + // + // Measured: `clang -target aarch64-linux-android21 -print-effective-triple` + // answers `aarch64-unknown-linux-android21`, so the level belongs on + // the ENV segment of the effective triple and nowhere else. It selects + // which bionic symbols are visible, which is why it is in the + // fingerprint; and it is the project's statement rather than the + // toolchain's, because one NDK serves a range of levels. + // + // ABSENT, THERE IS NO ANSWER, AND THIS COMMENT USED TO SAY OTHERWISE. + // It read "clang's own default applies [...] so omitting the key is a + // legal answer and not a gap", which was never verified. Measured: + // + // --target=aarch64-unknown-linux-android (no level) + // sys/cdefs.h:365:2: error: Unversioned target triples are not + // supported! + // + // bionic refuses an unversioned triple outright, so the level is + // mandatory. The project still does not have to state it: mcpp reads + // the NDK's own declared floor from `meta/platforms.json` and uses that + // (see `min_platform_version` and `ndk_min_api_level`), which is the + // payload answering for itself rather than a constant compiled in. + if (os == "linux") { + std::string e = env.empty() ? std::string("gnu") : env; + if (env == "android" && !minPlatformVersion.empty()) + e += std::string(minPlatformVersion); + return arch + "-unknown-linux-" + e; + } + // Emscripten's own effective triple. The vendor segment is `unknown` + // and the OS segment is the platform layer rather than a kernel, which + // is why `object_format()` reads the ARCH for this row. + if (os == "emscripten") return arch + "-unknown-emscripten"; if (os == "none") return str(); // freestanding: already LLVM's form return str(); } @@ -124,7 +226,69 @@ struct Triple { bool is_musl() const { return env == "musl"; } bool is_msvc_env() const { return env == "msvc"; } bool is_windows_gnu() const { return os == "windows" && env == "gnu"; } - bool is_pe() const { return os == "windows"; } + + // THE SINGLE DERIVATION. Every question about the container objects come in + // is answered here and nowhere else -- see `ObjectFormat` for why a third + // value makes that a requirement rather than a preference. + // + // The arch test precedes the ELF fallback because a wasm target's OS + // segment names a platform layer (`emscripten`), not a format, and the + // fallback would otherwise claim ELF for it -- the silent wrong answer this + // whole axis exists to remove. + ObjectFormat object_format() const { + if (os == "windows") return ObjectFormat::Pe; + if (os == "macos" || os == "ios") return ObjectFormat::MachO; + if (arch.starts_with("wasm")) return ObjectFormat::Wasm; + return ObjectFormat::Elf; + } + + // Kept as its own name because it is what 30-odd sites already ask, and now + // reads the single answer rather than re-deriving one. + bool is_pe() const { return object_format() == ObjectFormat::Pe; } + bool is_mach_o() const { return object_format() == ObjectFormat::MachO; } + bool is_wasm() const { return object_format() == ObjectFormat::Wasm; } + + // DOES THIS TARGET'S TOOLCHAIN ARRIVE WITH ITS OWN COMPLETE SYSTEM? + // + // mcpp assembles a target's system for most rows: a glibc-targeting build + // gets `xim:glibc` and `xim:linux-headers` reconstructed onto the command + // line by hand, because the payload's clang alone does not have them. For + // `emscripten` and `android` that reconstruction is not merely unnecessary, + // it is WRONG -- both SDKs ship a complete sysroot and resolve it + // themselves (`em++` bakes `--sysroot=/.../cache/sysroot` into + // every invocation; the NDK's clang derives its bionic sysroot from its own + // install prefix). + // + // Measured before this predicate existed: `mcpp build --target + // wasm32-emscripten` failed inside the std module precompile, at + // + // .../sysroot/include/c++/v1/cstdint:149: + // .../xim-x-glibc/2.44/include/stdint.h:26 + // + // -- the HOST's glibc headers pulled into a wasm compile. The site that + // did it asked `is_freestanding()`, which is a correct question about the + // rows it was written for and says nothing about this one: a wasm target + // is not freestanding, it simply is not this host. + // + // A PROPERTY OF THE TOOLCHAIN, KEYED ON THE TARGET, and the two coincide + // because the target decides the payload (see registry.cppm's + // to_xim_package). If a row ever gains a second toolchain that does NOT + // ship a sysroot, this has to move onto the toolchain. + bool has_own_sysroot() const { + return os == "emscripten" || os == "android" || env == "android"; + } + + // APPLE, AS ONE QUESTION. `os == "macos"` was the whole of it while macOS + // was the only Apple row; iOS shares the object format, the linker, the + // `arm64` spelling and `codesign`, and differs in the SDK and the + // deployment-target flag. A site that means "Apple" and asks "macOS" gets + // iOS wrong in the direction that still links. + bool is_apple() const { return os == "macos" || os == "ios"; } + // Android is Linux with a different C library and a different loader path. + // `os` stays `linux` for that reason -- it is the kernel, and every + // Linux-shaped decision in the tree is right about it -- and the env + // segment carries what differs. + bool is_android() const { return env == "android"; } // Bare metal: there is no OS to link against. THE predicate every // freestanding decision keys off, spelled once here so no consumer @@ -151,12 +315,50 @@ struct Triple { // added later — was a convention at both. Measured: declaring gcc for it // resolved the host's Linux musl payload and reported a missing C++ // frontend. - bool pin_is_capability() const { return is_freestanding() || (is_pe() && is_musl()); } + // IS THIS ROW'S PIN A CAPABILITY STATEMENT RATHER THAN A CONVENTION? + // + // A convention pin is mcpp's preference and a declared toolchain overrides + // it. A capability pin cannot be overridden, because no other toolchain + // can emit the target at all: only clang/lld cross-compile bare metal, no + // gcc emits a PE with a musl C library, and -- added with the wasm row -- + // nothing but Emscripten emits WebAssembly. A declared `gcc@16.1.0` + // against such a row is a request that cannot be honoured, and saying so + // is better than resolving gcc and failing inside it. + // + // ANDROID BELONGS HERE FOR A REASON THE OTHER THREE DO NOT SHOW, and it + // was left out when the row got its pin. The other three are refused + // because the toolchain CANNOT emit the format; a stock clang emits + // aarch64 ELF perfectly well, so nothing about the output rules it out. + // What it cannot supply is the SYSTEM: bionic's headers, its per-API-level + // stubs and its loader path live inside the NDK, and there is no package + // that adds them to another compiler. + // + // Measured on the path a declared `llvm@22.1.8` takes without this: + // clang resolves, and the build stops inside it with + // `'__config' file not found`, then -- once told the target -- + // `Unversioned target triples are not supported!` from bionic's own + // . Two diagnostics, neither naming the toolchain that cannot + // serve the row. `-D__BIONIC_CTYPE_INLINE=` and an API level fused onto + // the triple are the NDK's own requirements, not flags a user can be + // expected to supply to a different compiler. + bool pin_is_capability() const { + return is_freestanding() || (is_pe() && is_musl()) || is_wasm() + || is_android(); + } // cfg() `family` dimension: unix | windows. + // + // iOS and Android are unix for the reason macOS and Linux are: the + // predicate answers about the API surface a source can assume, and both are + // POSIX. Emscripten is unix on the same test rather than on a claim about + // wasm -- it supplies a POSIX emulation, and a source guarded by + // `cfg(unix)` compiles against it. A target with no OS still answers + // nothing, unchanged: `cfg(unix)` on bare metal would be false in a way no + // source could act on. std::string family() const { if (os == "windows") return "windows"; - if (os == "linux" || os == "macos") return "unix"; + if (os == "linux" || os == "macos" || os == "ios" + || os == "emscripten") return "unix"; return {}; } @@ -413,6 +615,160 @@ inline constexpr TargetInfo kKnownTargets[] = { // library for these targets arrives from the dependency graph. { "armv7a-none-eabi", "verified", "bare","llvm@22.1.8","", true }, { "armv7a-none-eabihf", "verified", "bare","llvm@22.1.8","", true }, + + // ── The three platforms a package cannot add ──────────────────────────── + // + // A package can add a language, a tool, an action, a payload and a + // generated module. IT CANNOT ADD A TRIPLE: identity is these three + // strings and this table is compiled into the binary, so every layer above + // -- the `.apk` step, the `.app` step, the `.html`+`.wasm` step, the + // runner, the signing -- waits on a row here and on nothing else in the + // engine. Registering the rows is what turns each of those into a plugin + // that can be written rather than a plugin that has nowhere to attach. + // + // ALL FOUR ARE `planned`, WHICH IS A REFUSAL AND NOT A GAP. The tier gate + // refuses a planned row with `tier-planned` naming the row, so + // `mcpp build --target aarch64-linux-android` says the vocabulary has this + // target and nothing is wired yet -- rather than `unknown target`, which + // was false, or a build that resolves and produces nothing, which would be + // worse than either. What each row still needs is recorded in + // .agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md + // section 3, and it is a payload in every case, never engine work. + // + // THE PREREQUISITE NOBODY LISTS IS ANSWERED FOR TWO OF THE THREE. mcpp is + // module-first, so a row whose toolchain cannot compile a module interface + // unit would be worse than its absence. Measured 2026年09月11日: `import std` + // works on both the NDK's clang 18 and Emscripten's, and neither needs a + // fork or a compiler upgrade -- what both need is the generated module + // surface their vendor chose not to install (133 files, 620 KB, taken from + // the libc++ revision matching `_LIBCPP_VERSION`, which for Emscripten is + // NOT the version its clang reports). Apple's half is not measurable on a + // Linux host and is the one genuinely open question of the three. + + // ANDROID IS THE SMALLEST OF THE THREE, and the ranking is the opposite of + // the demand ranking. `aarch64` is already an arch, ELF is already the + // object format, and Linux is already the OS: what was missing is an `env` + // value and a sysroot that points at an NDK. No `pin`, because no cross + // payload exists yet -- `xim:android-ndk` is the row's whole remaining + // cost, and until it lands `[target.
].sysroot` is the escape hatch + // for a machine that has an NDK already. + { "aarch64-linux-android", "preview", "", "android-ndk@30.0.16248370", "", false }, + // The emulator's row, and the one of the pair that could be EXECUTED. + // + // Not a convenience: x86_64 is what an Android emulator image runs, so a + // row for the device without one for the emulator describes a target + // nothing can execute. That argument is now measured rather than asserted. + // 2026年09月11日, linux-x86_64, an API 24 x86_64 system image under the + // platform's own emulator with KVM: + // + // adb push /data/local/tmp/ + // adb shell ./andtest -> 1-2-3 exit 0 + // + // from `import std;` and no project vocabulary beyond `--target`. So this + // row is `verified` while `aarch64-linux-android` is `preview`, and the + // difference is execution rather than confidence in the build. + // + // WHY THE DEVICE ROW COULD NOT FOLLOW, recorded so the next attempt does + // not repeat it. Google's emulator refuses a foreign guest outright -- + // "QEMU2 emulator does not support arm64 CPU architecture" -- so the arm64 + // image needs an arm64 host. The documented fallback is qemu-user with the + // system image's own bionic, and preparing it needs four files extracted + // from an ext4 partition image by `debugfs`, which is the one program in + // `xim:e2fsprogs@1.47.3` that is a broken build (SIGFPE on every + // filesystem-opening command, while dumpe2fs/e2fsck/tune2fs from the same + // payload work). That is an ecosystem defect with its own record in the + // index, not an engine gap, and it moves this row to `verified` when it is + // fixed -- nothing here changes. + // + // One linker warning is worth recording because a user will see it and it + // is not a defect: `unsupported flags DT_FLAGS_1=0x8000001`. API 24's + // bionic linker does not recognise the `DF_1_PIE` bit that lld sets, warns, + // and loads the program anyway. + { "x86_64-linux-android", "verified", "", "android-ndk@30.0.16248370", "", false }, + + // iOS IS NEXT. `aarch64-macos` is `verified`, so Mach-O, `arm64`, the + // linker and the Apple half of the toolchain model all exist; what is + // missing is an `os` value and the iPhoneOS SDK. + // + // THE SDK IS A LICENCE QUESTION AND NOT A PACKAGING ONE, which is why this + // row carries no `sysroot`. The NDK is Apache-2.0 and Emscripten is MIT, + // both redistributable; the iPhoneOS SDK is neither. The recipe should + // reach for the lowest of three tiers its licence allows -- redistribute, + // fetch from upstream without a mirror, or locate what the machine already + // has -- and say which tier it took, because a consumer reading "locator" + // needs to know that is a licence conclusion rather than an unfinished + // recipe. `msvc@system` is the shape of the third tier and mcpp already + // has it. + // + // The simulator is deliberately not a row. It has its own SDK and produces + // its own object, so folding it in would make two targets share an + // identity -- the mistake `x86_64-windows-musl` was added to undo. + { "aarch64-ios", "planned", "", "", "", false }, + // THE SIMULATOR'S TWO ROWS. Not a convenience and not a runner: a + // simulator build has its own SDK (`iPhoneSimulator.sdk`), produces its own + // object, and takes `-mios-simulator-version-min` rather than + // `-miphoneos-version-min`. Two targets sharing one identity is what the + // device row's own comment objected to, and the objection was to the + // absence of these rows rather than to their presence. + // + // BOTH ARCHES, for the reason the Android pair has both: the simulator runs + // the HOST's architecture, so an Apple-silicon machine needs `aarch64` and + // an Intel one needs `x86_64`. A single row would describe a simulator half + // the machines cannot run. + // + // `planned`, and the blocker is the same licence question as the device + // row -- the simulator SDK ships inside Xcode and is no more + // redistributable than the iPhoneOS one. What these rows buy today is that + // `mcpp build --target aarch64-ios-sim` answers `tier-planned` naming the + // row, instead of `unknown target`, which was false. + { "aarch64-ios-sim", "planned", "", "", "", false }, + { "x86_64-ios-sim", "planned", "", "", "", false }, + + // WEB IS THE OUTLIER, AND IT IS THE ONLY ONE OF THE THREE THAT CHANGES THE + // MODEL RATHER THAN EXTENDING A TABLE. A new arch (`wasm32`), a new os + // (`emscripten`), and -- the sharp part -- a new OBJECT FORMAT, which + // before `ObjectFormat` existed was not a field at all but a derivation + // repeated at every site that needed it. That is why this is + // https://github.com/mcpp-community/mcpp/issues/597 and not a table row. + // + // IT IS NOW ONLY THAT, AND THE STANDARD-LIBRARY HALF IS SIMPLER THAN THIS + // COMMENT FIRST SAID. Measured 2026年09月11日 against Emscripten 6.0.9: + // `em++` compiles and links `import std` with NO additional flags and no + // generated surface at all, because the toolchain SHIPS one -- 134 files + // -- and `node app.js` printed the expected output. + // + // The version numbers here were two releases stale, in exactly the + // direction the design record warns about: they said llvm 20.1.7 and + // `_LIBCPP_VERSION 200100` against clang 22.0.0git. Emscripten 6.0.9 + // reports `220108` (llvm 22.1.8) and clang 24.0.0git. The rule those + // numbers were supporting is unaffected and is the reason to keep them + // accurate: the surface must match the LIBRARY, never the compiler, and a + // recipe's job is to pin the `_LIBCPP_VERSION` it measured and refuse a + // change. A stale number in a comment becomes a stale number in a + // diagnostic, and then in somebody's install command. + // + // `defaultStatic` is true because wasm has no dynamic loader in the sense + // the other rows mean: an Emscripten link produces one module plus its + // JavaScript, and there is no shared object for a search path to find. + // + // `verified` ASSERTS THE WHOLE LOOP, and here is what it was measured + // against (2026年09月11日, Linux x86_64, xim:emsdk 6.0.9): + // + // mcpp build --target wasm32-emscripten on a source that imports std + // -> bin/ 65389 bytes the JavaScript + // bin/.wasm 447183 bytes the module + // node bin/ -> 1-2-3 + // + // Reaching it took five engine gates, and each one was found by the + // previous one's failure rather than by reading: the payload had to be + // chosen by the TARGET (registry.cppm), the frontend found outside `bin/` + // (frontendSubdir), the host's header set withheld (has_own_sysroot, in + // the shared producer and not at one of its three callers), the C-runtime + // group withheld from the LINK MODEL rather than from its two channels, + // and -- the same finding a second time -- the COMPILER's own runtime + // directories kept off the ARTIFACT's link line. For every row that + // predates this one those two are the same directory. + { "wasm32-emscripten", "verified", "wasm","emsdk@6.0.9","", true }, }; inline std::span known_targets() { return kKnownTargets; } @@ -488,6 +844,34 @@ inline RequestResolution resolve_request(const Triple& parsed) { && k.canonical.starts_with(prefix) && k.canonical[prefix.size()] == '-'; if (!exact && !sub) continue; + // A BARE `arch-os` ASKS FOR A C LIBRARY TO BE FILLED IN, AND ANDROID + // IS NOT ONE OF THE ANSWERS. + // + // The candidates here are meant to be alternatives for the SAME + // platform -- `gnu` or `musl` for a Linux -- so that `aarch64-linux` + // can complete to the one this repository supports. `android` sits on + // the same `arch-os` prefix because its kernel IS Linux, which is the + // modelling decision that makes every Linux-shaped answer in the tree + // right about it; it is not an alternative C library for the same + // platform. It has a different loader path, a different SDK and an API + // level. + // + // It became visible the moment the Android rows stopped being + // `planned`: `aarch64-linux` then had TWO supported siblings and + // resolved as ambiguous, where before it completed to + // `aarch64-linux-musl`. Either outcome of an ambiguity would be wrong + // here -- refusing a request that has an obvious answer, or answering + // it with bionic. + // + // Excluded from `siblings` too, not just from `supported`. That list is + // what the diagnostic prints, and offering `aarch64-linux-android` to + // someone who typed `aarch64-linux` would be a suggestion to build for + // a different platform. + // + // A written `aarch64-linux-android` never reaches this loop: an + // explicit env returns above, which is the rule that an author's own + // spelling is a request and not a gap. + if (auto kt = parse(k.canonical); kt && kt->is_android()) continue; r.siblings.push_back(k.canonical); if (k.tier != "planned") r.supported.push_back(k.canonical); } @@ -762,6 +1146,19 @@ std::optional
parse(std::string_view s) { // "mingw32" is the GNU os segment for ALL MinGW targets (64-bit // included — historical residue); it means windows + gnu env. if (starts_with(k, "mingw")) { t.os = "windows"; sawOs = true; t.env = "gnu"; t.envExplicit = true; continue; } + // APPLE'S SECOND OS. `starts_with` for the same reason the macOS + // branch above uses it: an effective triple carries the deployment + // target on this segment (`arm64-apple-ios17.0`). The simulator is a + // different row and is deliberately not spelled here -- it has a + // different SDK and a different object, so folding it into this one + // would make two targets share an identity. + if (starts_with(k, "iphoneos") || starts_with(k, "ios")) + { t.os = "ios"; sawOs = true; t.env.clear(); continue; } + // EMSCRIPTEN IS AN OS SEGMENT, NOT AN ENV. It names the platform layer + // a wasm module is compiled against -- its POSIX emulation, its + // filesystem shim, its `main` loop -- which is the same kind of thing + // `linux` names and not the same kind of thing `musl` names. + if (starts_with(k, "emscripten")) { t.os = "emscripten"; sawOs = true; t.env.clear(); continue; } // Bare-metal object-format / ABI segments. Only meaningful with // os=none: `riscv64-none-elf`, `arm-none-eabi`, `arm-none-eabihf`. @@ -773,20 +1170,59 @@ std::optional
parse(std::string_view s) { } if (t.os != "macos") { + // ANDROID IS AN ENV SEGMENT ON A LINUX OS, and that placement is + // the whole of the modelling decision. The kernel IS Linux, so + // every Linux-shaped answer in the tree -- ELF, the `unix` family, + // `nasm -f elf64` -- is already right; what differs is the C + // library (bionic), the loader path and the SDK. An `os = "android"` + // would have made all three of those wrong by default and required + // a new answer at each site. + // + // `androideabi` is the 32-bit ARM spelling and resolves to the same + // env: the EABI half is the ARM calling convention, which `armv7a` + // already carries in the arch segment. + // + // AND THE API LEVEL RIDES THIS SEGMENT, SO THE MATCH HAS TO BE A + // PREFIX. This read `k == "android"`, which cannot parse + // `aarch64-unknown-linux-android21` -- a string mcpp PRINTS + // itself, one line above the build it describes: + // + // Target aarch64-linux-android -> aarch64-unknown-linux-android21 + // + // A reader who pastes that back was told mcpp had never heard of + // it. The msvc branch below already carries the identical note for + // the identical reason ("...-windows-msvc19.44.35211"); Android has + // the same shape and was missed. One prefix covers all four + // spellings: `android`, `android21`, `androideabi`, + // `androideabi21`. + if (starts_with(k, "android")) { + t.env = "android"; t.envExplicit = true; continue; + } if (k == "musl" || starts_with(k, "musleabi")) { t.env = "musl"; t.envExplicit = true; continue; } if (k == "gnu" || starts_with(k, "gnueabi")) { t.env = "gnu"; t.envExplicit = true; continue; } // starts_with: clang effective triples can carry a version suffix // on the env segment ("...-windows-msvc19.44.35211"). if (starts_with(k, "msvc")) { t.env = "msvc"; t.envExplicit = true; continue; } } - // Unrecognized segment (androideabi, wasi, ...): not in mcpp's target - // language — treat as unparseable rather than guessing. + // THE SIMULATOR SEGMENT, WHICH IS THE ONE PLACE AN APPLE ROW HAS AN + // env. Both spellings arrive: `sim` is mcpp's and Rust's, `simulator` + // is Apple's own and appears in any effective triple clang prints. + if (t.os == "ios" && (k == "sim" || k == "simulator")) { + t.env = "sim"; t.envExplicit = true; continue; + } + // Unrecognized segment (wasi, ...): not in mcpp's target language — + // treat as unparseable rather than guessing. return std::nullopt; } if (!sawOs) return std::nullopt; - // macOS carries no env segment at all, so nothing was declined there. - if (t.os == "macos") { t.env.clear(); t.envExplicit = false; } + // macOS carries no env segment at all, so nothing was declined there. iOS + // and Emscripten are the same shape: the platform layer is the whole of the + // identity past the arch, and there is no C-library axis to decline. + if (t.os == "macos" || t.os == "emscripten" + || (t.os == "ios" && t.env != "sim")) { + t.env.clear(); t.envExplicit = false; + } // THE FILL STAYS, AND THE FACT THAT IT WAS A FILL IS NOW RECORDED. // `x86_64-linux` is the canonical identity `x86_64-linux-gnu` — every // directory name and cache key downstream depends on that — but it is NOT diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 6a73bf867..9d997683e 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.10.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.11.3"; } // namespace mcpp diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index bd30891a7..90758a08a 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -130,6 +130,47 @@ struct BuildProgramEnv { // way to ask. See hostprogram::package_name for what it replaced. std::string packageName; std::string packageNamespace; + // THE REST OF `[package]`, FOR THE MEMBER OF THE COLLECTION THAT NEEDS IT. + // + // A rule generates a declaration and needs the package's NAME. A member + // that produces a DISTRIBUTABLE needs more: every installer format carries + // a version, and most carry a description, a licence and a maintainer. + // Without these a project has to restate them in the member's options, + // where they can drift from `[package]` with nothing able to detect it -- + // the second copy of a value whose first copy mcpp has already parsed. + // + // `packageAuthors` is joined with ';' rather than ',' because an author + // entry is conventionally `Name ` and a name may carry a comma. + // Empty under an engine that predates these, which a member reads as "fall + // back to whatever you did before". + std::string packageVersion; + std::string packageDescription; + std::string packageLicense; + std::string packageAuthors; + std::string packageRepo; + // ── The packaging pass this build is part of (mcpp 2026年9月11日.1+) ──────── + // + // Empty for every ordinary build, and that is the value that carries the + // meaning: a member which produces a distributable SUBMITS NOTHING unless + // the format it provides was asked for. `mcpp build` therefore has the + // graph it always had, and the dist edge exists only in the pass that + // wants it. + // + // The value is the `--format` argument verbatim -- `tar`, `dir`, or a name + // a package provides. It rides the same env vector as everything else here, + // so `contract_hash` folds it into the build program's re-run key: the + // second pass re-runs exactly the programs whose answer this changes. + std::string packFormat; + // Where `mcpp pack` has ALREADY STAGED the closure, absolute. Non-empty + // only in the second pass, and only then because a staged tree is produced + // by mcpp after the link -- so a graph generated before the link cannot + // name a directory that does not exist yet. + // + // This is the value `${mcpp.stage_dir}` expands to. A member reads it to + // decide the shape of the work (which of `bin/`, `lib/`, `share/` the tree + // actually has) and writes the placeholder into the action, so the two + // never disagree. + std::filesystem::path packStageDir; // Whether this package builds C++ modules (`[language] modules`). // // Reported because a rule package that GENERATES a consumer-facing @@ -530,6 +571,13 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_MANIFEST_DIR", root.string()); e.emplace_back("MCPP_PKG_NAME", env.packageName); e.emplace_back("MCPP_PKG_NAMESPACE", env.packageNamespace); + e.emplace_back("MCPP_PKG_VERSION", env.packageVersion); + e.emplace_back("MCPP_PKG_DESCRIPTION", env.packageDescription); + e.emplace_back("MCPP_PKG_LICENSE", env.packageLicense); + e.emplace_back("MCPP_PKG_AUTHORS", env.packageAuthors); + e.emplace_back("MCPP_PKG_REPO", env.packageRepo); + e.emplace_back("MCPP_PACK_FORMAT", env.packFormat); + e.emplace_back("MCPP_PACK_STAGE_DIR", env.packStageDir.string()); std::string csv; for (auto const& f : env.features) { if (!csv.empty()) csv += ','; @@ -1209,13 +1257,37 @@ std::expected run_build_program( auto hm = build_host_module(bdir, hostCompiler, base, std_flag, tc, compileEnv, ref.logical, ref.interface, use); if (!hm) return std::unexpected(hm.error()); - for (auto& f : hm->useFlags) { - // GCC's marker is just `-fmodules`, already present when the - // bundled module was built; repeating it is harmless but noisy. - if (std::find(moduleFlags.begin(), moduleFlags.end(), f) - == moduleFlags.end()) - moduleFlags.push_back(f); - } + // APPENDED VERBATIM, and nothing is de-duplicated. + // + // This filtered per TOKEN, and it was written for the one family whose + // marker is a single idempotent word: GCC's `-fmodules`, already + // present because the bundled `mcpp` module put it there. The other + // two families do not have that shape. + // + // GCC `-fmodules` 1 token, idempotent + // Clang `-fmodule-file==` 1 token, unique + // MSVC `/reference`, `=` 2 tokens, FIRST REPEATS + // + // So on `windows = "msvc@system"` the pair arrived, `/reference` was + // found already in the list, and only the pair's second half was + // appended. cl.exe received `=.ifc` with no switch in + // front of it and read it as a source file name: + // + // c1xx: fatal error C1083: Cannot open source file: + // 'huxerui.rules.sources=...\huxerui.rules.sources.ifc' + // + // Clang was immune by construction -- one word, never equal to an + // existing element -- which is why the defect was specific to the one + // toolchain selection that reaches `import std;` at c++20 on Windows. + // + // The comment this replaces stated the whole value of the filter: + // "repeating it is harmless but noisy". It bought a tidier argv and + // paid with a broken command line. De-duplicating by logical module + // name, or by contiguous subsequence, would both be correct -- and + // would both be a new rule kept for the same cosmetic reason. The rule + // is gone instead. + for (auto& f : hm->useFlags) + moduleFlags.push_back(f); hostModuleObjects.push_back(std::move(hm->object)); } @@ -1286,6 +1358,24 @@ std::expected run_build_program( } else { compileArgv.push_back("-o"); compileArgv.push_back(bin.string()); } + // A `=` with no switch in front of it, checked before the + // command runs rather than diagnosed from cl.exe's answer to it. cl reports + // such a token as `C1083: Cannot open source file`, which names the module + // and the BMI and never names the missing flag -- so it reads as a broken + // build tree rather than as a broken command line. See + // mcpp::toolchain::orphaned_reference; this is the reader that makes the + // rule enforced rather than merely stated. + if (auto orphan = mcpp::toolchain::orphaned_reference(compileArgv)) { + return std::unexpected(std::format( + "build.mcpp: the module reference '{}' reached the compiler with no " + "switch in front of it.\n" + " This is an mcpp defect, not a problem with the project: the " + "reference is\n" + " assembled as a pair (`/reference =` on MSVC) and " + "only one half\n" + " arrived. Please report it with the toolchain name and this " + "line.", *orphan)); + } mcpp::ui::info("build.mcpp", "compiling"); // GCC resolves imported BMIs via gcm.cache/ relative to the compile cwd, so // any compile that imports a module — `mcpp`, `std`, or both — has to run diff --git a/src/build/cache_key.cppm b/src/build/cache_key.cppm index 3222086e5..1bc42321c 100644 --- a/src/build/cache_key.cppm +++ b/src/build/cache_key.cppm @@ -135,7 +135,18 @@ struct BuildAxes { std::string cppStandardFlag; std::vector dialectFlags; std::string cStandard; - std::string macosDeploymentTarget; + // THE OLDEST OS RELEASE THIS ARTEFACT MUST RUN ON, one slot for both + // platforms that have such a thing. + // + // It was `macosDeploymentTarget`. Android's minimum API level is the same + // quantity -- and it is in this key for the same reason: it selects which + // bionic symbols are visible, so two levels are two ABIs and must never + // share a build directory. A target is either Apple or Android, so one + // slot cannot be asked to hold both at once. + // + // Renaming the hashed label costs no extra rebuild: the mcpp version is + // already part of this key, so every release invalidates it anyway. + std::string minPlatformVersion; // C std::string optLevel; bool debug = false; @@ -208,7 +219,7 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, const mcpp::manifest::Manifest& rootManifest, std::string_view cppStandardFlag, const std::vector& dialectFlags, - std::string_view macosDeploymentTarget, + std::string_view minPlatformVersion, const std::filesystem::path& storeRoot = {}, bool needsPic = false); @@ -273,7 +284,7 @@ nlohmann::json to_json(const BuildAxes& b, const PackageAxes& p) { {"cpp_standard_flag", b.cppStandardFlag}, {"dialect_flags", b.dialectFlags}, {"c_standard", b.cStandard}, - {"macos_deployment_target", b.macosDeploymentTarget}, + {"min_platform_version", b.minPlatformVersion}, }; j["profile"] = { {"opt_level", b.optLevel}, @@ -321,7 +332,7 @@ std::string key_hex(const BuildAxes& b, const PackageAxes& p) { put(s, "stdflag", b.cppStandardFlag); put_list(s, "dialect", b.dialectFlags); put(s, "cstd", b.cStandard); - put(s, "macos", b.macosDeploymentTarget); + put(s, "minplat", b.minPlatformVersion); // C put(s, "opt", b.optLevel); put(s, "debug", b.debug ? "1" : "0"); @@ -353,7 +364,7 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, const mcpp::manifest::Manifest& rootManifest, std::string_view cppStandardFlag, const std::vector& dialectFlags, - std::string_view macosDeploymentTarget, + std::string_view minPlatformVersion, const std::filesystem::path& storeRoot, bool needsPic) { @@ -468,7 +479,7 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, b.cppStandardFlag = std::string(cppStandardFlag); b.dialectFlags = dialectFlags; b.cStandard = rootManifest.buildConfig.cStandard; - b.macosDeploymentTarget = std::string(macosDeploymentTarget); + b.minPlatformVersion = std::string(minPlatformVersion); b.optLevel = rootManifest.buildConfig.optLevel; b.debug = rootManifest.buildConfig.debug; diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index 52ca0d694..b269c831a 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -101,7 +101,7 @@ enum class Contract { // The binary format decides which mechanisms even exist — Mach-O has no // priority-ordered initializer section, PE has no rpath, ELF has both. -enum class Format { Elf, MachO, Pe }; +enum class Format { Elf, MachO, Pe, Wasm }; // WHICH FORMAT A TARGET PRODUCES, ASKED OF THE TARGET. // @@ -129,9 +129,29 @@ enum class Format { Elf, MachO, Pe }; Format format_for(std::string_view targetTriple, Format hostFallback) { if (auto parsed = mcpp::toolchain::triple::parse(targetTriple)) { if (parsed->is_pe()) return Format::Pe; - if (parsed->os == "macos") return Format::MachO; + if (parsed->is_wasm()) return Format::Wasm; + // `is_mach_o()`, not `os == "macos"`: the latter answered the + // opposite-hosts defect above for macOS and would still get iOS + // wrong the same way, since iOS's `os` is `ios`. + if (parsed->is_mach_o()) return Format::MachO; if (parsed->os == "linux" || parsed->os == "none") return Format::Elf; + // THE FOURTH MEMBER, and this comment used to say it was deferred + // "to whoever gives this module a mechanism for it". A wasm target + // parses here and fell out of every branch -- `is_pe()` and + // `is_mach_o()` are both false, and its `os` is `emscripten`, neither + // `linux` nor `none` -- reaching `hostFallback` and answering the + // MACHINE's format, which is the defect class this header measured + // for macOS. + // + // What it cost while it stood: every wasm build printed + // + // warning: cxx_runtime: distributable target: this toolchain ships + // no libc++.a/libc++abi.a; using toolchain-coupled (the artifact + // keeps a run-time dependency on the toolchain's libc++.so) + // + // -- a promise about a `libc++.so` that cannot exist for this target, + // on an artifact that has no run-time dependency of any kind. } if (targetTriple.find("windows") != std::string_view::npos || targetTriple.find("mingw") != std::string_view::npos) @@ -608,6 +628,26 @@ Mechanism resolve(const MechanismInput& in) { return m; } + // -------------------------------------------------------------- WASM + // + // THERE IS NOTHING TO BE COUPLED TO. An Emscripten link produces one + // module plus its JavaScript: no `DT_NEEDED`, no rpath, no loader, no + // shared object a search path could find. So the artifact is + // self-contained by construction rather than by flags, and the contract + // is satisfied with nothing added -- which is also why there is no + // degradation to report. A diagnostic here would be a broken promise + // about a mechanism the format does not have. + // + // `-nostdlib++` and the archive pair are deliberately NOT emitted. libc++ + // reaches a wasm link through `em++`'s own link line (`-lc++-debug-noexcept + // -lc++abi-debug-noexcept`, measured), and naming archives from a sysroot + // this module did not resolve would be the second answer to a question the + // driver has already answered. + case Format::Wasm: { + m.effective = Contract::SelfContained; + return m; + } + // --------------------------------------------------------------- ELF case Format::Elf: default: { diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 8d3b9b5e4..0ca19e7d0 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -2176,7 +2176,16 @@ export int run_tests(std::span passthrough, // until the test ended — including when it hangs, which is exactly when a // reader needs it. const int runJobs = [&] { - int j = mcpp::build::schedule::resolve_jobs(ctx->manifest); + // The machine's `[build] default_jobs` applies HERE TOO, and that is a + // decision rather than an inheritance. A test runner at ten concurrent + // processes has the same memory shape as a compile at ten, so someone + // who set a machine-wide number almost certainly meant it for both; + // `docs/04-mcpp-toml.md` says so, because one key with two + // behaviours has to be stated. The fallback below is unchanged: + // absent, this path uses the whole machine rather than the backend's + // default, since there is no backend to defer to. + int j = mcpp::build::schedule::resolve_jobs(ctx->manifest, {}, + ctx->globalDefaultJobs); if (j <= 0) j = static_cast(std::thread::hardware_concurrency()); return j> 0 ? j : 1; }(); diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 7c2a64c18..2394eb8c3 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -531,6 +531,13 @@ CompileFlags compute_flags(const BuildPlan& plan) { const std::string crossTarget = plan.toolchain.crossTargetFlag.empty() ? std::string{} : " " + plan.toolchain.crossTargetFlag; + // Does the TARGET bring its own sysroot -- an Emscripten or Android SDK, + // where the C library, the C++ runtime and the loader are all inside the + // payload? Read once here; the link branch below is its only consumer. + const bool ownSysrootTarget = [&] { + auto tt = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + return tt && tt->has_own_sysroot(); + }(); const bool isClangWithCfg = dm.hasCfg; // THE TARGET SIDE COMES FROM THE DEPENDENCY GRAPH, READ RATHER THAN @@ -586,6 +593,14 @@ CompileFlags compute_flags(const BuildPlan& plan) { return ft && ft->is_freestanding(); }(); + // A TARGET WHOSE TOOLCHAIN SHIPS ITS OWN SYSROOT IS HANDLED IN THE SHARED + // PRODUCER, not here. `host_compile_tokens` is read by this site, by the + // std module's command assembly (stdmod.cppm) and by the build.mcpp host + // compile, and the first attempt at this fix put the answer at THIS site + // only -- so `mcpp build --target wasm32-emscripten` stopped injecting the + // host's headers into ordinary compiles and went on injecting them into the + // std module precompile, which is where it had been failing. One decision, + // one site, three readers. if (!isFreestandingTarget) { mcpp::toolchain::HostFlagOptions hopt; hopt.cfgBypass = mcpp::toolchain::HostFlagOptions::CfgBypass::Always; @@ -680,6 +695,32 @@ CompileFlags compute_flags(const BuildPlan& plan) { link_toolchain_flags = crossTarget + lm.link_flags(ninjaEsc); link_toolchain_flags_c = link_toolchain_flags; // nothing C++-only here f.sysroot = link_toolchain_flags; + } else if (!crossTarget.empty() && ownSysrootTarget) { + // AN SDK THAT BRINGS ITS OWN SYSROOT STILL HAS TO BE TOLD WHICH TARGET. + // + // Both branches above are skipped for such a target, and that is + // correct for everything they carry: the link model contributes + // nothing, because the C library, the C++ runtime, the crt objects and + // the loader all live inside the SDK and the driver finds them itself. + // What it cannot do is guess WHICH of them to find -- one NDK serves + // both Android arches -- so falling through with an empty string linked + // the target's objects with the host's startup files: + // + // hermetic link check failed + // /lib/x86_64-linux-gnu/Scrt1.o (outside the sandbox) + // /usr/lib/gcc/x86_64-linux-gnu/13/crtbeginS.o + // /lib64/ld-linux-x86-64.so.2 + // + // Six host objects on an aarch64 link, every one of them resolved by a + // driver that believed it was building for this machine. The compile + // side already said the target; only the link side did not. + // + // `crossTarget` ALONE, and that is the whole content of this branch. + // Adding the C-runtime flags the branch above adds would reintroduce + // the host's model, which is the thing the SDK replaces. + link_toolchain_flags = crossTarget; + link_toolchain_flags_c = crossTarget; + f.sysroot = link_toolchain_flags; } // Binutils -B flag — a GCC/libstdc++ payload concern (musl and MinGW-w64 @@ -691,6 +732,40 @@ CompileFlags compute_flags(const BuildPlan& plan) { const auto linkIntentFlavor = [&] { if (isMingwTc) return LinkIntentFlavor::PeGnu; if (isMsvcDialect) return LinkIntentFlavor::PeMsvc; + // THE OBJECT FORMAT IS ASKED OF THE PARSED TRIPLE, AND THE SUBSTRING + // TEST BELOW IS NOW ONLY THE ESCAPE HATCH. + // + // `plan.toolchain.targetTriple` is mcpp's CANONICAL spelling, and + // `aarch64-macos` contains neither "apple" nor "darwin" -- so an + // explicit `--target aarch64-macos`, a verified row, fell through to + // `Elf`. Only a NATIVE macOS build was right, and by a different + // branch: an empty triple reaching the `needs_explicit_libcxx` rescue + // below. That is why nothing caught it -- the two paths through this + // function disagreed and only one of them was exercised. + // + // The substring test is kept for a triple `parse` REJECTS, which is + // the `[target.
]` escape hatch: an author may name a spelling + // outside the canonical vocabulary, and it is then an LLVM-shaped + // string where "apple" and "windows" do appear. + if (auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple)) { + switch (t->object_format()) { + case mcpp::toolchain::triple::ObjectFormat::MachO: + return LinkIntentFlavor::MachO; + case mcpp::toolchain::triple::ObjectFormat::Pe: + return LinkIntentFlavor::PeGnu; + case mcpp::toolchain::triple::ObjectFormat::Wasm: + // No `LinkIntentFlavor::Wasm` exists, and inventing one + // here would be a link-contract decision rather than a + // format question -- what `link_lib` and a search path + // even mean for an Emscripten link is the open half of + // #597. `Elf` is the wrong answer and is the one this + // returns; it is named here so the gap is visible rather + // than reached by falling off the end of a switch. + return LinkIntentFlavor::Elf; + case mcpp::toolchain::triple::ObjectFormat::Elf: + return LinkIntentFlavor::Elf; + } + } auto triple = plan.toolchain.targetTriple; std::ranges::transform(triple, triple.begin(), [](unsigned char c) { return std::tolower(c); }); diff --git a/src/build/graph_shape.cppm b/src/build/graph_shape.cppm index f8581c8e7..679d2d043 100644 --- a/src/build/graph_shape.cppm +++ b/src/build/graph_shape.cppm @@ -62,11 +62,22 @@ std::string_view to_string(GraphShape shape) { // `mcpp build --no-accel`, `mcpp build` -- the third reported "Finished in // 0.00s" and `mcpp run` executed the CPU variant. A graph an override chose // says so, and the fast paths, which run only without overrides, decline it. +// `packFormat` records the DISTRIBUTION FORMAT this graph was generated for, +// and empty means "none" -- an ordinary build. It rides this line for exactly +// the reason the other two fields do, and it is the third instance of one +// failure: `mcpp pack --format appimage` makes a build program submit an +// artifact action a plain build must not have, and the two graphs land in the +// same directory because the format is deliberately NOT in the fingerprint +// (putting it there would cost a full recompile to package an already-built +// tree). So `pack --format X` then `build` would replay a graph carrying a dist +// edge, which is the `A then B then A` shape both other fields exist to stop. std::string header_line(GraphShape shape, std::string_view scheduleTag, - bool accelOverridden = false) { - return std::format("# mcpp:graph={};schedule={};accel={}", + bool accelOverridden = false, + std::string_view packFormat = {}) { + return std::format("# mcpp:graph={};schedule={};accel={};dist={}", to_string(shape), scheduleTag, - accelOverridden ? "override" : "default"); + accelOverridden ? "override" : "default", + packFormat.empty() ? std::string_view("none") : packFormat); } // Read the shape back. `nullopt` means "this file does not say" — a build.ninja @@ -162,13 +173,39 @@ std::string read_accel_selection(const std::filesystem::path& ninjaPath) { return {}; } -// A graph the fast paths may replay: the package's own targets, and the -// device variant the manifest names rather than one a flag chose. Both fast -// paths run only when no override is present, so a graph an override wrote is -// never the graph a plain build would produce. +// The distribution format this graph was generated for, or "none". Empty when +// the file predates the field, which callers treat as a miss for the reason +// read_shape gives. +std::string read_pack_format(const std::filesystem::path& ninjaPath) { + std::ifstream input(ninjaPath); + if (!input) return {}; + std::string line; + for (int i = 0; i < 8 && std::getline(input, line); ++i) { + constexpr std::string_view prefix = "# mcpp:graph="; + if (!line.starts_with(prefix)) continue; + auto value = std::string_view(line).substr(prefix.size()); + while (!value.empty() && (value.back() == '\r' || value.back() == ' ')) + value.remove_suffix(1); + constexpr std::string_view key = ";dist="; + const auto at = value.find(key); + if (at == std::string_view::npos) return {}; + auto rest = value.substr(at + key.size()); + if (const auto semi = rest.find(';'); semi != std::string_view::npos) + rest = rest.substr(0, semi); + return std::string(rest); + } + return {}; +} + +// A graph the fast paths may replay: the package's own targets, the device +// variant the manifest names rather than one a flag chose, and no distribution +// edge. All three fast-path callers run only for a plain build, so a graph any +// of the three axes was pointed at is never the graph a plain build would +// produce. bool is_plain_build_graph(const std::filesystem::path& ninjaPath) { return read_shape(ninjaPath) == GraphShape::Normal - && read_accel_selection(ninjaPath) == "default"; + && read_accel_selection(ninjaPath) == "default" + && read_pack_format(ninjaPath) == "none"; } } // namespace mcpp::build diff --git a/src/build/hermetic.cppm b/src/build/hermetic.cppm index e192bfcbe..43e7eb0fb 100644 --- a/src/build/hermetic.cppm +++ b/src/build/hermetic.cppm @@ -28,6 +28,7 @@ import mcpp.log; import mcpp.platform; import mcpp.toolchain.fingerprint; import mcpp.toolchain.model; +import mcpp.toolchain.triple; export namespace mcpp::build { @@ -186,7 +187,31 @@ std::expected verify_hermetic_link( auto base = std::filesystem::path(std::string(t)).filename().string(); if (is_crt_object(base)) check(t); } - if (!effectiveLoader.empty()) check(effectiveLoader); + // THE LOADER OF A TARGET THAT BRINGS ITS OWN SYSROOT IS A PATH ON THE + // TARGET, NOT ON THIS MACHINE. + // + // Every other path this function inspects is resolved by the linker here + // and must therefore sit inside a payload. The dynamic linker is the one + // that is not: it is recorded in the artefact and read by the DEVICE at + // load time. Android's is `/system/bin/linker64` by ABI -- it cannot be + // inside a payload, and an artefact naming a payload path there would be + // the defect rather than the proof. + // + // Measured: with the crt objects and libc++ correctly resolving inside the + // NDK, this was the single remaining "leak" and the build stopped on it. + // The message was accurate about what it saw and wrong about what it meant, + // which is the harder kind: it named a real path outside the sandbox and + // invited the reader to reinstall a glibc payload that has nothing to do + // with it. + if (auto tt = mcpp::toolchain::triple::parse(tc.targetTriple); + tt && tt->has_own_sysroot()) { + mcpp::log::verbose("hermetic", std::format( + "target {} carries its own sysroot; its loader ({}) is a path on " + "the target and is not checked against the sandbox", + tc.targetTriple, effectiveLoader)); + } else if (!effectiveLoader.empty()) { + check(effectiveLoader); + } if (!leaks.empty()) { std::string list; diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 88e21b8d6..fed7c862b 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -124,6 +124,33 @@ inline void fact(const char* name, const char* version) { } inline void floor(const char* spec) { std::printf("mcpp:floor=%s\n", spec); } +// ── The distributable channel (mcpp 2026年9月11日.1+) ─────────────────────────── +// +// `mcpp pack --format ` dispatches to whichever package provides +// ``, exactly as `--target` reaches a triple the engine did not have to +// know individually. This is how a package says which name it answers for: +// +// mcpp::provides_pack_format("appimage"); +// +// DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. The declaration must not be +// gated on `pack_format()`, and the reason is that the engine has to be able to +// answer a question the requesting build cannot: `mcpp pack --format bogus` +// names what IS available, and `--help` says "plus any format the resolved +// graph provides". Both read the set collected from this outlet, on a build +// that asked for nothing. A member that declared only when asked would still +// work for its author -- they always pass their own format -- and would make +// the set unknowable for everyone else. +// +// The work itself is the other half: +// +// if (std::string_view(mcpp::pack_format()) == "appimage") { ... submit ... } +// +// Two names are reserved for the engine's own archive shapes and are refused +// here: `tar` and `dir`. +inline void provides_pack_format(const char* name) { + std::printf("mcpp:pack-format=%s\n", name); +} + // The memory layout for a freestanding link. Reaches the CONSUMER's link line // (like link_lib/link_search, unlike include_dir), because the package that // knows a board's layout is not the package being built. @@ -390,6 +417,53 @@ inline const char* manifest_dir() { return env_or("MCPP_MANIFEST // rule package working unchanged. inline const char* package_name() { return env_or("MCPP_PKG_NAME"); } inline const char* package_namespace() { return env_or("MCPP_PKG_NAMESPACE"); } + +// THE REST OF `[package]`, BECAUSE A DISTRIBUTABLE CARRIES IT. +// +// `package_name()` above exists so a generated declaration can be named. These +// exist for the other member of the collection: every installer format states a +// version, and most state a description, a licence and a maintainer. A member +// without them has to ask the PROJECT to restate values mcpp has already +// parsed, in the member's own options, where the copy drifts from `[package]` +// and nothing can detect that it has. +// +// `package_authors()` is a ';'-separated list -- not ',', because an author is +// conventionally `Name ` and a name may carry a comma. +// +// Empty under an engine older than 2026年9月11日.1. A member that needs one must +// say so itself when it is empty, naming the value it wanted: only the member +// knows whether the absence is fatal. +inline const char* package_version() { return env_or("MCPP_PKG_VERSION"); } +inline const char* package_description() { return env_or("MCPP_PKG_DESCRIPTION"); } +inline const char* package_license() { return env_or("MCPP_PKG_LICENSE"); } +inline const char* package_authors() { return env_or("MCPP_PKG_AUTHORS"); } +inline const char* package_repo() { return env_or("MCPP_PKG_REPO"); } + +// WHICH DISTRIBUTABLE THIS PASS WAS ASKED FOR, or "" for every ordinary build. +// +// The empty value is the one that carries the meaning: a member gates its +// submission on this, so `mcpp build` has the graph it always had and a dist +// edge exists only in the pass that wants one. See `provides_pack_format` for +// the half that must NOT be gated. +inline const char* pack_format() { return env_or("MCPP_PACK_FORMAT"); } + +// WHERE `mcpp pack` HAS ALREADY STAGED THE CLOSURE, absolute; "" when this +// build is not packing. +// +// The tree is what `mcpp pack` computes and then, until this existed, threw +// away: the dependency closure after the strip policy, the debug-symbol split +// and `include`/`exclude`. It is a BUNDLE tree -- `bin/`, `lib/`, relocatable, +// rooted anywhere -- which is what an AppImage, a `.app` and an `.msi` want as +// it stands. A format that wants a root filesystem instead (`.deb`, `.rpm`) +// owns the re-layout, because which directory a file belongs in is that +// format's knowledge and not the engine's. +// +// READ IT HERE TO DECIDE, WRITE `${mcpp.stage_dir}` INTO THE ACTION. The +// directory exists while this program runs, so a member enumerates it to learn +// which of `bin/`, `lib/`, `share/` the tree actually has; the action's command +// then names it through the placeholder, so the path in the graph and the path +// this program read cannot disagree. +inline const char* pack_stage_dir() { return env_or("MCPP_PACK_STAGE_DIR"); } inline bool has_feature(const char* name) { char buf[256] = "MCPP_FEATURE_"; unsigned long o = 13; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 4ff138ce9..cdebeeef0 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -260,15 +260,18 @@ std::string pe_link_flag(const BuildPlan& plan, bool sep, std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { if (lu.kind != LinkUnit::SharedLibrary) return ""; const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); - const std::string os = t ? t->os - : (mcpp::platform::is_macos ? "macos" - : mcpp::platform::is_windows ? "windows" : "linux"); + // WHICH FLAG SPELLING, ASKED OF THE OBJECT FORMAT DIRECTLY rather than of + // an `os` string: the prior `os == "macos"` fell through to the ELF + // branch (`-Wl,-soname`, a GNU ld/BFD flag ld64 does not accept) for + // iOS, whose `os` is `ios`. + const bool pe = t ? t->is_pe() : bool(mcpp::platform::is_windows); + const bool macho = t ? t->is_mach_o() : bool(mcpp::platform::is_macos); // PE records no such name: a DLL is found by the filename in the importing // module's import table, and there is nothing to override. - if (os == "windows") return ""; + if (pe) return ""; const std::string name = lu.soname.empty() ? lu.output.filename().string() : lu.soname; - if (os == "macos") return "-Wl,-install_name,@rpath/" + name; + if (macho) return "-Wl,-install_name,@rpath/" + name; return lu.soname.empty() ? "" : "-Wl,-soname," + lu.soname; } @@ -291,7 +294,11 @@ std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { // script's syntax is not what the author wrote. std::string exports_file_contents(const LinkUnit& lu, std::string_view os) { std::string out; - if (os == "macos") { + // Which SYMBOL-TABLE SYNTAX, which is a property of the object format: + // iOS links with the same ld64 and the same leading-underscore Mach-O + // symbol table as macOS, so it takes this branch too rather than the + // GNU version-script one below, which ld64 does not parse. + if (os == "macos" || os == "ios") { // One symbol per line. Mach-O symbols carry a leading underscore that // the C++ source never writes, so it is added here -- the author names // the symbol, not the object format's spelling of it. @@ -316,7 +323,10 @@ std::string exports_flag(const LinkUnit& lu, std::string_view os, const std::filesystem::path& file) { if (lu.kind != LinkUnit::SharedLibrary || lu.exportPatterns.empty()) return ""; if (os == "windows") return ""; - if (os == "macos") + // iOS alongside macOS, matching `exports_file_contents`: same linker, + // same flag. Leaving it out sent an iOS shared-library link a GNU + // `--version-script` for a Mach-O symbol list, which ld64 rejects. + if (os == "macos" || os == "ios") return "-Wl,-exported_symbols_list," + file.generic_string(); return "-Wl,--version-script=" + file.generic_string(); } @@ -626,7 +636,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { // write this one file and the fast path has to know what it is about to // replay. Must stay within the first few lines — see read_shape. append(mcpp::build::header_line(plan.graphShape, plan.scheduleTag, - plan.accelOverridden) + "\n"); + plan.accelOverridden, + plan.packFormat) + "\n"); append("ninja_required_version = 1.11\n\n"); // All compile/link flags are computed once via flags.cppm. diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 34433a1f8..0fe1db43d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -270,6 +270,20 @@ struct BuildPlan { // absolute and engine variables already substituted by the time they get // here, so the backend only has to spell edges. std::vector actions; + // The distribution formats the RESOLVED GRAPH provides, sorted and unique + // (`mcpp:pack-format=`). This is what `mcpp pack --format ` resolves + // against and what an unknown value's refusal names. + // + // Collected on EVERY prepare, including the pass that asked for no format + // at all -- which is the pass that has to answer "what is available". See + // `mcpp::provides_pack_format` for the author-facing rule that makes this + // possible: declare unconditionally, submit conditionally. + std::vector providedPackFormats; + // Non-empty when this prepare is the second pass of `mcpp pack --format + // `: the staged closure `${mcpp.stage_dir}` expanded to. Recorded on + // the plan so the graph header line can say which format wrote this graph, + // and the fast paths can decline to replay it for a plain build. + std::string packFormat; std::vector runtimeLibraryDirs; // ONLY the dependency packages' [runtime] library_dirs (not toolchain/ // payload dirs). These are the dirs that must be baked into the produced @@ -541,8 +555,12 @@ std::vector shared_library_link_flags( const mcpp::toolchain::triple::Triple& target) { std::vector flags; const bool pe = n.sharedNeedsImportLib; + // WHICH RPATH SYNTAX, ASKED OF THE OBJECT FORMAT. `target.os == "macos"` + // used to answer this and missed iOS, which links with the same ld64 and + // wants the same `@loader_path` -- `os == "ios"` would otherwise take the + // ELF branch below and hand `$ORIGIN` to a linker that has no such token. const bool macho = target.empty() ? bool(mcpp::platform::is_macos) - : target.os == "macos"; + : target.is_mach_o(); if (pe) { flags.push_back(import_library_for(t, n).generic_string()); } else { @@ -796,9 +814,16 @@ std::vector runtime_search_closure( auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); return t ? *t : mcpp::toolchain::triple::Triple{}; }(); + // `!= "macos" && != "windows"` READ AS "ELF" BY EXCLUSION, WHICH IS THE + // ONE ANSWER THAT MUST NEVER BE REACHED BY EXCLUDING EVERYTHING ELSE: + // iOS and wasm32-emscripten both satisfy that double negative (their `os` + // is `ios` / `emscripten`, neither string), so this line called an iOS + // Mach-O and a wasm module ELF and would have handed both a DT_RPATH + // mechanism neither format has. Asked of `object_format()` directly, the + // single derivation, instead. const bool elfTarget = triple.empty() ? bool(mcpp::platform::is_linux) - : (triple.os != "macos" && triple.os != "windows"); + : (triple.object_format() == mcpp::toolchain::triple::ObjectFormat::Elf); // THE ARTIFACT'S OWN DIRECTORY — `$ORIGIN` (#415). // @@ -1045,10 +1070,13 @@ make_plan(const mcpp::manifest::Manifest& manifest, // The loader-tag contract exists only where DT_RPATH/DT_RUNPATH do. // Mach-O and PE have neither, so they get no flag rather than a branch in - // every consumer. + // every consumer. Asked of `object_format()`: the exclusion form this + // used to be (`!= "macos" && != "windows"`) answers "ELF" for any `os` it + // does not name, which is wrong the same way for iOS and for + // wasm32-emscripten — see the sibling derivation earlier in this file. const bool elfTarget = targetTriple.empty() ? bool(mcpp::platform::is_linux) - : (targetTriple.os != "macos" && targetTriple.os != "windows"); + : (targetTriple.object_format() == mcpp::toolchain::triple::ObjectFormat::Elf); auto loader_tag_flag = [&](LinkUnit::Kind kind) -> std::string { if (!elfTarget) return {}; using mcpp::build::loader::Form; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 69b9b47fb..b4b3fa1a5 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -61,6 +61,7 @@ import mcpp.build.runtime_validation; // declared artifact -> identity verdict import mcpp.build.cache_key; import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against import mcpp.pack.prebuilt; // ...and the check itself +import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest import mcpp.build.build_program; import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides @@ -766,6 +767,17 @@ export std::string_view cache_mode_name(CacheMode m) { } export struct BuildContext { + // THE PER-MACHINE JOB DEFAULT, carried so it is read once. + // + // `[build] default_jobs` in `$MCPP_HOME/config.toml` is the machine's + // answer to "how many at once". `prepare_build` resolves it into the build + // schedule itself; this field exists for the SECOND reader -- + // `mcpp test`'s runner concurrency (execute.cppm) -- which calls + // `resolve_jobs` again after this function has returned. Recorded rather + // than re-read, because a second `load_or_init` there would be a second + // parser of one file, and because the two readers must not be able to + // disagree about the machine. + int globalDefaultJobs = 0; // --strict: degradations reported through mcpp::diag become errors. // Carried on the context because the build's degradations are discovered // during backend emission, i.e. after prepare_build has returned — the @@ -998,6 +1010,34 @@ export struct BuildOverrides { // One says which PACKAGES enter the graph, the other which TOOLS are // installed, and `mcpp run` needs the second without the first. bool will_run = false; + // ── The packaging pass, when this prepare is one (mcpp 2026年9月11日.1+) ──── + // + // `mcpp pack --format ` prepares TWICE, and these two fields are the + // whole difference between the passes. The first sets neither: build + // programs run, declare the formats they provide, and submit no dist + // action because none was asked for. The second sets both, after the link + // and after staging, so the claiming member submits an action whose input + // is a directory that by then exists. + // + // NEITHER VALUE IS DERIVED HERE, AND THAT IS THE POINT. `pack_stage_dir` is + // a function of the package name, the version, the resolved triple and the + // mode, and the resolved triple is not known until a prepare has run. + // Computing it a second time before prepare -- from the host triple, say -- + // is the shape where two derivations of one value agree on every machine + // the author has. Both are read out of what the first pass and `make_plan` + // already answered. + std::string pack_format; + std::filesystem::path pack_stage_dir; + // WHY THERE IS NO STAGED TREE, when there is none and a format was still + // requested. Empty otherwise. + // + // A dispatched format does not require the built-in bundling to have + // succeeded -- see the note in `mcpp.pack.pipeline`. When it did not, the + // reason travels here so `${mcpp.stage_dir}`'s refusal can name it instead + // of saying only that the placeholder is unavailable. A member author + // reading "this build is not packaging" for a build that plainly is would + // be sent looking in the wrong place. + std::string pack_stage_reason; }; // ── git dependency helpers ────────────────────────────────────────────────── @@ -1134,6 +1174,34 @@ mcpp::platform::process::RunResult run_with_network_retry( return r; } +// `[package]`, for the build program of the package that declares it. +// +// ONE CALL RATHER THAN A FIELD PER SITE. Two places build a +// `BuildProgramEnv` -- the dependency loop and the root -- and the values a +// build program is told about its own package are the same question in both. +// Setting them field by field at each site is how the two answers drift: the +// root gained `packageName` and the dependency loop gained it separately, and +// a value added to only one of them is a rule package that works for a root +// project and not for a dependency, with nothing failing to say so. +void fill_package_build_env(mcpp::build::BuildProgramEnv& e, + const mcpp::manifest::Manifest& m) +{ + e.packageName = m.package.name; + e.packageNamespace = m.package.namespace_; + e.packageVersion = m.package.version; + e.packageDescription = m.package.description; + e.packageLicense = m.package.license; + e.packageRepo = m.package.repo; + // ';' rather than ',': an author entry is conventionally `Name ` + // and a name may carry a comma, so a comma-joined list cannot be split back + // into the entries it was made from. + e.packageAuthors.clear(); + for (auto const& a : m.package.authors) { + if (!e.packageAuthors.empty()) e.packageAuthors += ';'; + e.packageAuthors += a; + } +} + void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::toolchain::Toolchain* tc) { @@ -1491,6 +1559,45 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, return {}; } +// THE PROJECT'S MINIMUM PLATFORM VERSION FOR THIS TARGET, in one place. +// +// Two platforms fuse it into the effective triple and each names it in its own +// words: macOS's deployment target lives in `[build]` because it applies to +// every Apple artefact a project produces, and Android's API level lives in +// `[target.
]` because it applies to one row. `llvm_triple` takes one +// parameter for both, so the choice between them is made here rather than at +// each of its call sites -- there are two, and a decision made twice is the +// shape this codebase records most often. +std::string min_platform_version(const mcpp::manifest::Manifest& m, + const mcpp::toolchain::triple::Triple& t, + const std::filesystem::path& compilerPath) { + if (t.is_android()) { + if (auto it = m.targetOverrides.find(t.str()); it != m.targetOverrides.end()) + if (it->second.minApiLevel> 0) + return std::to_string(it->second.minApiLevel); + // AND THERE IS NO SUCH THING AS LEAVING IT OUT. This returned an empty + // string with the comment "the NDK's own default, which clang + // supplies", which was never verified and is false. Measured: + // + // --target=aarch64-unknown-linux-android (no level) + // sys/cdefs.h:365:2: error: Unversioned target triples are not + // supported! + // + // bionic refuses it, so the level is mandatory and a project that + // never heard of API levels still needs one. The NDK declares the + // floor it supports in `meta/platforms.json` and that is the honest + // default -- the payload's own answer, which moves when the payload + // does. macOS is the same shape and already works this way: its + // default comes from the platform module, not from the manifest. + if (auto level = mcpp::toolchain::ndk_min_api_level(compilerPath); + level> 0) + return std::to_string(level); + return {}; // the caller refuses; see android_api_level_refusal + } + return mcpp::platform::macos::deployment_target( + m.buildConfig.macosDeploymentTarget); +} + std::string with_index_cause(std::string msg) { if (auto hint = mcpp::pm::unusable_index_hint(); !hint.empty()) msg += "\n" + hint; @@ -2498,30 +2605,89 @@ prepare_build(bool print_fingerprint, if (known && parsed && parsed->pin_is_capability() && tc_origin_is_user_explicit(tcOrigin) && tcSpec.has_value()) { auto declared = mcpp::toolchain::parse_toolchain_spec(*tcSpec); - if (declared && declared->family != mcpp::toolchain::Family::Llvm) { - // THE REASON TRAVELS WITH THE ROW. Both rows refuse for the + // WHICH DECLARATIONS THE ROW ACCEPTS IS THE ROW'S PIN, NOT A FIXED + // FAMILY. + // + // This asked `family != Llvm`, which was right while every + // capability-pinned row pinned llvm. `wasm32-emscripten` pins + // `emsdk@6.0.9`, and emsdk NORMALISES to the llvm family -- `em++` + // is clang -- so a declared `llvm@22.1.8` passed this gate, was + // never refused, and resolved the generic llvm payload for a target + // it cannot emit. The condition is now the pin's own family, which + // is the question the row was always answering. + const auto pinFamily = [&]() -> std::optional { + if (known->pin.empty()) return mcpp::toolchain::Family::Llvm; + if (auto ps = mcpp::toolchain::parse_toolchain_spec( + std::string(known->pin))) + return ps->family; + return std::nullopt; + }(); + const bool declaredMatchesPin = + declared && pinFamily && declared->family == *pinFamily + // An emsdk row is llvm-family, so the family alone cannot + // separate `emsdk@6.0.9` from `llvm@22.1.8`. The pin's own + // spelling is what does. + && (known->pin.empty() + || tcSpec->find(known->pin.substr(0, known->pin.find('@'))) + != std::string::npos); + if (declared && !declaredMatchesPin) { + // THE REASON TRAVELS WITH THE ROW. The rows refuse for the // same rule and NOT for the same reason, and one sentence - // covering both would be wrong about one of them: a PE+musl - // target is not bare metal, and a reader told it is stops - // reading. + // covering all of them would be wrong about the others: a + // PE+musl target is not bare metal, a wasm target is neither, + // and a reader told the wrong one stops reading. + // + // Measured before the third arm existed: `--target + // wasm32-emscripten` with a declared gcc was refused correctly + // and explained with "No gcc payload emits a PE with a musl C + // library", which is a true sentence about a different row. + // + // IT HAPPENED AGAIN, AND ADDING AN ARM IS ONLY HALF THE FIX. + // Android became a capability row and this chain still had + // three arms, so a declared `llvm@22.1.8` against + // `aarch64-linux-android` was refused correctly and explained + // with the PE+musl sentence -- the identical wrong answer the + // paragraph above records for wasm, reached the same way: by a + // fourth case falling into a final `else` that was written as + // the third case's answer. + // + // So the last arm now NAMES ITS OWN ROW and the fallthrough is + // generic. A capability added later gets a sentence that is + // merely unspecific instead of one that is false, and the + // refusal still names the pin either way. std::string_view why = parsed->is_freestanding() ? "A freestanding target has no per-host cross payload: " "clang and lld are\n" " cross-compilers by construction and gcc is not." - : "No gcc payload emits a PE with a musl C library — the " + : parsed->is_wasm() + ? "Nothing but Emscripten emits WebAssembly: `em++` is a " + "clang whose target,\n" + " sysroot and JavaScript glue all come from its own " + "payload." + : parsed->is_android() + ? "An Android target needs bionic, not just an aarch64 or " + "x86_64 back end:\n" + " its headers, its per-API-level stubs and its " + "loader path are inside the\n" + " NDK, and no package adds them to another compiler." + : (parsed->is_pe() && parsed->is_musl()) + ? "No gcc payload emits a PE with a musl C library — the " "mingw payload emits\n" " PE with the MinGW CRT, which is the separate " - "`-gnu` row."; + "`-gnu` row." + : "This row's toolchain is the only one that can emit the " + "target at all."; refusal::record(refusal::Code::CapabilityPin); return std::unexpected(std::format( "target '{}' cannot be emitted by '{}'.\n" " {}\n" - " The row names llvm as a capability rather than as a " - "preference, so this\n" - " one line is not a convention you can override.\n" + " The row names `{}` as a capability rather than as a " + "preference, so\n" + " this one line is not a convention you can override.\n" " remove the `[toolchain]` line for this target, or set " "it to `{}`.", parsed->str(), *tcSpec, why, + known->pin.empty() ? std::string_view("llvm") : known->pin, known->pin.empty() ? std::string_view("llvm") : known->pin)); } } @@ -2771,11 +2937,12 @@ prepare_build(bool print_fingerprint, "{} → msvc {} ({})", spec->display(), inst->display_version(), inst->clPath.string())); } else { - explicit_compiler = mcpp::toolchain::toolchain_frontend(payload->binDir, pkg); + explicit_compiler = mcpp::toolchain::payload_frontend(payload->root, pkg); if (!std::filesystem::exists(explicit_compiler)) { return std::unexpected(std::format( "toolchain payload '{}' has no known C++ frontend in {}", - pkg.target(), payload->binDir.string())); + pkg.target(), + mcpp::toolchain::payload_frontend_dir(payload->root, pkg).string())); } // Same post-install fixup as `mcpp toolchain install` — this // manifest [toolchain] path previously ran none, so a freshly @@ -2997,11 +3164,12 @@ prepare_build(bool print_fingerprint, " mcpp toolchain install {}", defaultSpec, payload.error().message, defaultSpec)); } - explicit_compiler = mcpp::toolchain::toolchain_frontend(payload->binDir, defaultPkg); + explicit_compiler = mcpp::toolchain::payload_frontend(payload->root, defaultPkg); if (!std::filesystem::exists(explicit_compiler)) { return std::unexpected(std::format( "default toolchain payload {} has no known C++ frontend in {}", - defaultPkg.target(), payload->binDir.string())); + defaultPkg.target(), + mcpp::toolchain::payload_frontend_dir(payload->root, defaultPkg).string())); } // The freshly-installed toolchain needs the SAME post-install fixup @@ -3199,6 +3367,34 @@ prepare_build(bool print_fingerprint, { tc->targetTriple = want->str(); + // AND THE GATE THAT ALREADY EXISTS FOR THIS, APPLIED WHERE THE + // ANSWER IS KNOWN. + // + // `discover_link_runtime_dirs` refuses to report these + // directories for a target that carries its own sysroot, and the + // refusal never fired: that function runs during DETECTION, + // before this line, when `targetTriple` is still the HOST's. The + // gate read a host triple and answered correctly about it. + // + // The artefact is what showed it. An Android link line carried + // + // -L /toolchains/llvm/prebuilt/linux-x86_64/lib/ + // x86_64-unknown-linux-gnu + // + // whose last component is this machine's triple, produced by + // `root / "lib" / targetTriple` -- so the string names the + // question that was asked. Those are the compiler's own host + // runtime directories; an Android artefact must resolve libc++, + // the crt objects and the loader from the NDK's sysroot, and the + // hermetic check reported exactly that failure with six host + // objects. + // + // Cleared rather than re-derived. Re-running the discovery with + // the final triple would also change what every OTHER clang cross + // target gets, and those are measured as they stand; the claim + // being made here is only the one the gate already states. + if (want->has_own_sysroot()) tc->linkRuntimeDirs.clear(); + // And the flag that says it to the driver — for a HOSTED target // only. Freestanding already emits its own `--target`, together // with the ISA flags that must accompany it @@ -3208,8 +3404,55 @@ prepare_build(bool print_fingerprint, && tc->compiler == mcpp::toolchain::CompilerId::Clang) { tc->crossTargetFlag = "--target=" + want->llvm_triple( - mcpp::platform::macos::deployment_target( - m->buildConfig.macosDeploymentTarget)); + min_platform_version(*m, *want, tc->binaryPath)); + + // AND THE SAME FLAG ON THE std MODULE'S OWN COMMANDS, FOR A + // PAYLOAD THAT SERVES MORE THAN ONE TARGET. + // + // The std module is built by its own command assembly + // (clang.cppm), not by the compile flags, so a decision made + // only here reaches every translation unit and not that. For + // most toolchains the omission cannot be seen: a payload + // whose compiler IS its target finds its own headers, and a + // package-provided module carries the target inside + // `stdModuleFlags`. + // + // ONE NDK SERVES BOTH ANDROID ARCHES, which is the property + // that makes this necessary and is stated in the row's own + // pin: `android-ndk@` names no arch, so `--target` is the + // only thing that says which. Without it the precompile + // resolved libc++'s `#include <__config>` against the + // building machine and stopped there. + // + // NOT `has_own_sysroot()`, though both rows that answer true + // to it are SDKs with their own sysroot. Emscripten's `em++` + // serves exactly one target and needs no flag -- the verified + // wasm loop is measured without it -- so widening the gate to + // the predicate would add a flag to a command that does not + // want one. The property here is "one payload, several + // targets", and Android is the only row that has it; a future + // row brings its own measurement. + if (want->is_android()) { + tc->stdModuleTargetFlags = " " + tc->crossTargetFlag; + // BIONIC'S ctype HEADER AND A MODULE'S EXPORT RULES. + // + // bionic declares `isalnum` and its neighbours + // `static inline`, and libc++'s module surface exports + // them with `using std::isalnum`. A using-declaration + // cannot export a name with internal linkage, so the + // precompile fails on 14 names at once. Defining the + // macro empty makes those declarations extern, which is + // what every other C library this engine compiles + // against already does. + // + // Scoped to the std module and not to every unit: the + // rule being satisfied is about exporting from a module, + // and a translation unit that includes + // directly is entitled to bionic's inline definitions. + // `xim:android-ndk`'s own install-time self-test reaches + // the identical conclusion from the other direction. + tc->stdModuleTargetFlags += " -D__BIONIC_CTYPE_INLINE="; + } } } if (auto want = mcpp::toolchain::triple::parse(overrides.target_triple); @@ -3412,11 +3655,12 @@ prepare_build(bool print_fingerprint, pins::kSuggestGccMingw, pins::kFirstRunWinGnuTarget)); } explicit_compiler = - mcpp::toolchain::toolchain_frontend(payloadR->binDir, gnuPkg); + mcpp::toolchain::payload_frontend(payloadR->root, gnuPkg); if (!std::filesystem::exists(explicit_compiler)) { return std::unexpected(std::format( "MinGW-w64 payload {} has no known C++ frontend in {}", - gnuPkg.target(), payloadR->binDir.string())); + gnuPkg.target(), + mcpp::toolchain::payload_frontend_dir(payloadR->root, gnuPkg).string())); } if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( **cfgR, payloadR->root, gnuPkg, @@ -3590,11 +3834,12 @@ prepare_build(bool print_fingerprint, "host toolchain for build.mcpp ('{}'): {}", *tcSpec, payload.error().message)); } - auto frontend = mcpp::toolchain::toolchain_frontend(payload->binDir, pkg); + auto frontend = mcpp::toolchain::payload_frontend(payload->root, pkg); if (!std::filesystem::exists(frontend)) { return std::unexpected(std::format( "host toolchain payload '{}' has no known C++ frontend in {}", - pkg.target(), payload->binDir.string())); + pkg.target(), + mcpp::toolchain::payload_frontend_dir(payload->root, pkg).string())); } if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( **cfgH, payload->root, pkg, @@ -6542,7 +6787,7 @@ prepare_build(bool print_fingerprint, return std::unexpected(std::format( "`{}` requires the compiler to be `{}`, and mcpp has no " "compiler family by that name.\n" - " known families: gcc, llvm, msvc.", + " known families: gcc, llvm, msvc, emsdk, android-ndk.", reqCompilerBy, family)); } @@ -8198,8 +8443,9 @@ prepare_build(bool print_fingerprint, // The DECLARING package's setting, not the root project's: a rule // generating a declaration for this package must match how this // package is compiled. - bpEnv.packageName = pkg.manifest.package.name; - bpEnv.packageNamespace = pkg.manifest.package.namespace_; + fill_package_build_env(bpEnv, pkg.manifest); + bpEnv.packFormat = overrides.pack_format; + bpEnv.packStageDir = overrides.pack_stage_dir; bpEnv.languageModules = pkg.manifest.language.modules; bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(pkg.root.string()); dit != deviceSourcesByPackage.end()) @@ -8624,8 +8870,7 @@ prepare_build(bool print_fingerprint, if (tc) { if (auto tt = mcpp::toolchain::triple::parse(tc->targetTriple)) { in.llvmTriple = tt->llvm_triple( - mcpp::platform::macos::deployment_target( - m->buildConfig.macosDeploymentTarget)); + min_platform_version(*m, *tt, tc->binaryPath)); in.targetOs = tt->os; in.targetEnv = tt->env; in.freestandingTarget = tt->is_freestanding(); @@ -8881,8 +9126,39 @@ prepare_build(bool print_fingerprint, // out of order. Deferring the CHOICE the way the target side itself was // deferred is the structural fix and is its own change; until then the // user is told what happened and how to state the preference once. + // + // AND NOT FOR A ROW WHOSE PIN IS A CAPABILITY, WHERE BOTH HALVES OF + // THIS SENTENCE ARE FALSE. + // + // The warning says the default "would have served" the target and then + // tells the reader to declare it. On a capability row neither holds: + // nothing but the pinned payload can emit the target at all, and the + // declaration it suggests is REFUSED by the capability gate a few + // hundred lines above -- so following the advice replaces a warning + // with an error. + // + // Measured on `openkal-linux` built for `x86_64-linux-android`, whose + // target side does come from the graph: + // + // warning: ... so gcc@16.1.0 would have served x86_64-linux-android. + // State the preference: [target.x86_64-linux-android] + // toolchain = "gcc@16.1.0" + // $ (declaring exactly that) + // error: target 'x86_64-linux-android' cannot be emitted by + // 'gcc@16.1.0'. + // + // The first claim is false on its own terms too: this gcc payload + // cannot emit an Android object whatever the graph supplies. `graph` + // answers "who supplies the SYSTEM", and a capability pin answers "who + // can emit the FORMAT AND THE SYSTEM" -- two questions, and only the + // second one decides whether a substitution was avoidable. + const bool pinIsCapability = [&] { + auto tt = mcpp::toolchain::triple::parse(resolvedTargetCanonical); + return tt && tt->pin_is_capability(); + }(); if (!pinReplacedDefault.empty() - && resolvedTargetSide.system_from_graph()) { + && resolvedTargetSide.system_from_graph() + && !pinIsCapability) { mcpp::diag::warning("toolchain", std::format( "this project's target side comes from its dependency graph, so " "{} would have served {}.\n" @@ -9126,8 +9402,9 @@ prepare_build(bool print_fingerprint, bpEnv.toolsBin = projectSubosBin; bpEnv.profile = effectiveProfile; bpEnv.accel = resolvedAccel(); - bpEnv.packageName = m->package.name; - bpEnv.packageNamespace = m->package.namespace_; + fill_package_build_env(bpEnv, *m); + bpEnv.packFormat = overrides.pack_format; + bpEnv.packStageDir = overrides.pack_stage_dir; bpEnv.languageModules = m->language.modules; bpEnv.ruleModules = m->buildConfig.ruleModules; if (auto dit = deviceSourcesByPackage.find(root->string()); dit != deviceSourcesByPackage.end()) @@ -10179,6 +10456,27 @@ prepare_build(bool print_fingerprint, // fast path runs without overrides, so a graph written under one must not // be the graph it replays. ctx.plan.accelOverridden = !overrides.accel.empty(); + + // THE MACHINE'S JOB DEFAULT, resolved unconditionally and never fatally. + // + // `get_cfg` is lazy, so by this point the config may or may not have been + // loaded -- a project with no dependencies can reach here without touching + // it. Asking for it here rather than reading whatever `cfg_opt` happens to + // hold is the point: otherwise the same project would honour + // `[build] default_jobs` or ignore it depending on whether it has + // dependencies, which is an answer that depends on an unrelated axis. + // + // A failure is discarded. This value is a concurrency hint, and a build + // must not fail because the machine's preferred job count could not be + // read; every other consumer of the config already reports its own + // failures with a diagnostic that fits what it needed the config FOR. + // `requireBootstrap=false` because nothing here needs the bootstrap + // toolchain. + int globalDefaultJobs = 0; + if (auto c = get_cfg(/*requireBootstrap=*/false)) + globalDefaultJobs = static_cast((*c)->defaultJobs); + ctx.globalDefaultJobs = globalDefaultJobs; + // Resolve the module-edge schedule ONCE, here, where both the toolchain and // the manifest are in hand. The backend writes the graph in this shape, the // graph records the tag, and `mcpp build --verbose` prints the reason — all @@ -10197,7 +10495,7 @@ prepare_build(bool print_fingerprint, mcpp::build::schedule::resolve_jobs(*m, [](std::string_view bad) { mcpp::ui::warning(std::format( "ignoring invalid job count '{}' (expected a positive number or 'auto')", bad)); - }), + }, globalDefaultJobs), // What this machine would pick if asked. Impure, so it is resolved // here and handed to the pure `decide`. Only DetachCodegen uses it, // and only when the user gave no job count — without it that @@ -10242,7 +10540,32 @@ prepare_build(bool print_fingerprint, // become an edge with a blank path, and ninja reports that far away // from the typo that caused it. std::set unresolvedTargets; - auto substitute = [&](std::string s) { + // `${mcpp.stage_dir}` used where there is no staged tree, and used by an + // action whose role runs before the link. Both are refusals rather than + // empty expansions: an empty path is a token the command still accepts, + // and the tool then reads the build directory root -- which exists, so + // the mistake produces a plausible artifact instead of a diagnostic. + // Section 2 of the design record measured that shape: a valid, empty, + // 52 KB installer with nothing said about it. + std::set stageDirNoPass, stageDirWrongRole; + // Carried from the overrides so the refusal below can say WHY there is + // no tree, which is a different sentence from "you are not packaging". + std::string stageDirWhy; + // WHETHER *THIS* ACTION REFERENCED THE STAGED TREE, and deliberately a + // flag rather than a set keyed on the action's id: an id is unique + // within the package that declared it and nothing more, so two packages + // may each submit a `dist` action called `package`. A set would then + // hand one package's implicit dependency to the other's edge -- the + // shape where a predicate is right and the object is wrong, which does + // not fail, it answers about something else. + // + // The diagnostic sets below stay keyed by id because a diagnostic + // NAMES ids and a collision there costs a duplicate line, not a wrong + // edge. + bool thisActionUsesStageDir = false; + const bool stagePass = !overrides.pack_stage_dir.empty(); + auto substitute = [&](std::string s, const char* actionId, + mcpp::manifest::BuildAction::Role role) { auto rep = [&](std::string_view what, const std::string& with) { for (std::size_t p; (p = s.find(what)) != std::string::npos; ) s.replace(p, what.size(), with); @@ -10250,6 +10573,22 @@ prepare_build(bool print_fingerprint, rep("${mcpp.out_dir}", ctx.plan.outputDir.string()); rep("${mcpp.bin_dir}", (ctx.plan.outputDir / "bin").string()); rep("${mcpp.compile_db}", ctx.plan.compileDbPath.string()); + // ABSOLUTE, unlike `${mcpp.target_file:}` and for the same reason + // stated the other way round: the staged tree lives outside the + // build directory and no ninja edge produces it, so there is no + // edge-declared spelling to agree with. `${mcpp.out_dir}` above is + // absolute on the same grounds. + if (s.find("${mcpp.stage_dir}") != std::string::npos) { + if (!stagePass) { + stageDirNoPass.insert(actionId); + stageDirWhy = overrides.pack_stage_reason; + } else if (role != mcpp::manifest::BuildAction::Role::Artifact) { + stageDirWrongRole.insert(actionId); + } else { + thisActionUsesStageDir = true; + } + rep("${mcpp.stage_dir}", overrides.pack_stage_dir.string()); + } constexpr std::string_view kTf = "${mcpp.target_file:"; for (std::size_t p; (p = s.find(kTf)) != std::string::npos; ) { auto close = s.find('}', p); @@ -10278,22 +10617,86 @@ prepare_build(bool print_fingerprint, // mcpp#534's ordering edge is scoped to this name. auto owner = mcpp::build::qualified_package_name(mm); for (auto a : mm.buildConfig.actions) { - for (auto& x : a.inputs) x = substitute(x); - for (auto& x : a.outputs) x = substitute(x); - for (auto& x : a.command) x = substitute(x); + thisActionUsesStageDir = false; + const auto sub = [&](std::string v) { + return substitute(std::move(v), a.id.c_str(), a.role); + }; + for (auto& x : a.inputs) x = sub(x); + for (auto& x : a.outputs) x = sub(x); + for (auto& x : a.command) x = sub(x); // Same closed vocabulary as outputs — a depfile commonly // wants to live at `${mcpp.out_dir}/.d`, beside the // output it describes, and `prepare_actions` above // deliberately left a `${mcpp.` depfile untouched for // exactly this phase to resolve. - if (!a.depfile.empty()) a.depfile = substitute(a.depfile); + if (!a.depfile.empty()) a.depfile = sub(a.depfile); + // THE DEPENDENCY IS IMPLIED BY THE USE, so a member author + // cannot forget it. Without this the edge is dirty only when a + // link output changes, and a staged set that grew a dependency's + // shared library while the program's own bytes did not would + // leave the previous distributable in place, reported as + // up to date. + if (thisActionUsesStageDir) { + a.consumesStageDir = true; + a.inputs.push_back( + mcpp::pack::stage_manifest_path(overrides.pack_stage_dir).string()); + } a.packageName = owner; ctx.plan.actions.push_back(std::move(a)); } + // Every package's declaration, on every pass. Sorted and de-duplicated + // below so the refusal's list reads the same whatever order resolution + // walked the graph in. + for (auto const& f : mm.buildConfig.packFormats) + ctx.plan.providedPackFormats.push_back(f); }; collect(*m); for (std::size_t i = 1; i < packages.size(); ++i) collect(packages[i].manifest); + std::ranges::sort(ctx.plan.providedPackFormats); + ctx.plan.providedPackFormats.erase( + std::ranges::unique(ctx.plan.providedPackFormats).begin(), + ctx.plan.providedPackFormats.end()); + ctx.plan.packFormat = overrides.pack_format; + if (!stageDirNoPass.empty()) { + std::string ids; + for (auto const& n : stageDirNoPass) ids += (ids.empty() ? "" : ", ") + n; + if (!stageDirWhy.empty()) { + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}}, and no " + "tree could be staged for this target.\n" + " {}\n" + " The format was requested and the provider was reached; what is " + "missing is the staged\n" + " closure itself. A member that names a built file with " + "${{mcpp.target_file:}} instead\n" + " of reading the tree is unaffected on this target.", + ids, stageDirWhy)); + } + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}}, and this " + "build is not packaging.\n" + " The staged tree is produced by `mcpp pack` after the link, so " + "it does not exist during\n" + " a plain build and there is nothing for the placeholder to name.\n" + " Gate the submission on the format you provide:\n" + " mcpp::provides_pack_format(\"\"); // always\n" + " if (std::string_view(mcpp::pack_format()) == \"\") " + "// then submit\n" + " and reach the tree with `mcpp pack --format `.", ids)); + } + if (!stageDirWrongRole.empty()) { + std::string ids; + for (auto const& n : stageDirWrongRole) ids += (ids.empty() ? "" : ", ") + n; + return std::unexpected(std::format( + "build.mcpp action(s) [{}] reference ${{mcpp.stage_dir}} with a role " + "other than \"artifact\".\n" + " Only an artifact action runs after the link, and the staged tree " + "is a link output's\n" + " successor: a source, object or check action is scheduled before " + "there is anything to stage.\n" + " use: role = \"artifact\"", ids)); + } if (!unresolvedTargets.empty()) { std::string bad, known; for (auto const& n : unresolvedTargets) bad += (bad.empty() ? "" : ", ") + n; @@ -10801,8 +11204,15 @@ prepare_build(bool print_fingerprint, mcpp::toolchain::cppfly::effective_dialect_flags( *tc, m->cppStandard.experimental, mcpp::manifest::dialect_flags(m->buildConfig)), - mcpp::platform::macos::deployment_target( - m->buildConfig.macosDeploymentTarget), + // ONE SLOT, BOTH PLATFORMS. See `min_platform_version`: a target + // is either Apple or Android, and the level selects which bionic + // symbols are visible, so two levels must be two build + // directories. + [&] { + auto tt = mcpp::toolchain::triple::parse(tc->targetTriple); + return tt ? min_platform_version(*m, *tt, tc->binaryPath) + : std::string{}; + }(), // The GLOBAL registry root — the same one `fill_package_config` // relativizes against below, so both halves of the key describe // payload paths the same way. @@ -11384,14 +11794,29 @@ prepare_build(bool print_fingerprint, ctx.plan.runtimeBinding), nullptr, false); if (binding.is_discarded()) binding = nlohmann::json::object(); - auto triple = ctx.tc.targetTriple; - std::ranges::transform(triple, triple.begin(), - [](unsigned char c) { return std::tolower(c); }); - const bool pe = triple.find("windows") != std::string::npos - || triple.find("mingw") != std::string::npos; - const bool macho = triple.find("darwin") != std::string::npos - || triple.find("apple") != std::string::npos; - std::string format = pe ? "pe" : macho ? "macho" : "elf"; + // ASKED OF THE PARSED TRIPLE, with the substring test kept only for a + // spelling `parse` rejects. This field is the SECOND copy of a + // derivation `mcpp.build.dist::format_for` already owns, and it had + // the same defect: mcpp's canonical `aarch64-macos` contains neither + // "apple" nor "darwin", so an explicit `--target aarch64-macos` + // recorded `"elf"` while the native build on the same machine recorded + // `"macho"` -- one report contradicting the other about one machine. + std::string format = "elf"; + if (auto t = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) { + format = std::string(mcpp::toolchain::triple::to_string(t->object_format())); + std::ranges::transform(format, format.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (format == "mach-o") format = "macho"; + } else { + auto triple = ctx.tc.targetTriple; + std::ranges::transform(triple, triple.begin(), + [](unsigned char c) { return std::tolower(c); }); + const bool pe = triple.find("windows") != std::string::npos + || triple.find("mingw") != std::string::npos; + const bool macho = triple.find("darwin") != std::string::npos + || triple.find("apple") != std::string::npos; + format = pe ? "pe" : macho ? "macho" : "elf"; + } // The ORDERED run-time search closure with provenance. Order is // semantics here, not presentation: it is what the loader will walk, // and the mutable SubOS farm sitting last is the invariant that keeps @@ -11410,7 +11835,7 @@ prepare_build(bool print_fingerprint, } nlohmann::json search = { {"format", format}, - {"link_library", pe ? "libpath" : "library_path"}, + {"link_library", format == "pe" ? "libpath" : "library_path"}, {"transitive_needed", format == "elf" ? "rpath_link" : "none"}, {"runtime", format == "pe" ? "deploy" : format == "macho" ? "loader_rpath" : "runpath"}, diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index cc2cd1d18..95278c157 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -26,6 +26,7 @@ import mcpp.runtime.elf; import mcpp.runtime.binding; import mcpp.ui; import mcpp.platform.runtime_search; +import mcpp.toolchain.triple; export namespace mcpp::build::runtime_validation { @@ -612,6 +613,34 @@ ValidationReport validate_changed_artifacts( plan.runtimeBinding.runtimeId) != "glibc") return report; + // AND THE ARTIFACT HAS TO BE ONE THAT COULD LOAD ON THIS MACHINE. + // + // Every rule below compares an artifact against `plan.runtimeBinding` -- + // the loader, the libc and the search order of a process on THIS host. + // That premise is what makes the rules true, and it is false for a target + // whose system comes from inside an SDK: an Android executable's + // `PT_INTERP` is `/system/bin/linker64` by ABI and is read by the device. + // + // Measured on a correct artifact -- + // + // ELF 64-bit LSB pie executable, ARM aarch64, interpreter + // /system/bin/linker64 + // + // -- rule B called it a proven defect, because the host binding selects + // this machine's `ld-linux-x86-64.so.2` and "one process cannot mix + // runtime payloads" is a true sentence about a process that will never + // exist. It then offered a SubOS as the fix, which cannot help. The + // preceding two checks in this build had the same shape and each was + // corrected where its own premise lives; this is the third and last. + // + // The linux/glibc guard above does not cover it: an Android triple has + // `os == "linux"` on purpose -- it IS the kernel -- so every Linux-shaped + // decision in the tree is right about it except the ones that mean "this + // machine". + if (auto tt = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + tt && tt->has_own_sysroot()) + return report; + auto doc = read_cache(plan.outputDir); bool changedCache = false; if (doc.value("schema", 0) != 1 @@ -1248,6 +1277,33 @@ check_dlopen_surface(const mcpp::build::BuildPlan& plan) { return report; } + // AND NO SURFACE MEANS NOTHING TO READ. THE CHEAP TEST COMES FIRST. + // + // `inspect_dlopen_surface` walks `plan.depRuntimeLibraryDirs`. With that + // list empty there are no members, so every reading below is taken to + // answer a question with no subject -- and the readings are not cheap: the + // SONAME scan that follows parses EVERY linked artifact in full. + // + // Measured on this repository, 2026年09月11日, before this guard existed: + // + // mcpp build loader-tags stage 170ms one 21 MB binary + // mcpp test loader-tags stage 17948ms 108 binaries, 2.4 GB + // + // and flat at ~17.9s across every target measured, because the cost is the + // whole artifact set rather than the one being built. `mcpp test` drives + // the backend once per target, so a 110-target suite paid it 110 times -- + // turning a 3-minute run into 33. The record it produced every time said + // `members=0, walked=0`. + // + // This is the shape worth naming: the expensive work ran BEFORE the cheap + // test that makes it unnecessary. The record is still published, because a + // field that disappears is worse than a field that says why it is empty. + if (plan.depRuntimeLibraryDirs.empty()) { + publish_reason("no dependency published a runtime library directory; " + "there is no dlopen surface to judge"); + return report; + } + for (auto const& [artifact, stamp] : artifacts) { auto dir = artifact.parent_path(); if (dir.empty() || std::ranges::find(searchDirs, dir) != searchDirs.end()) @@ -1258,8 +1314,25 @@ check_dlopen_surface(const mcpp::build::BuildPlan& plan) { // The SONAMEs this build produces, read from the objects rather than from // their filenames. See `inspect_dlopen_surface` for why a filename search // is not enough while the build is still running. + // ONLY A SHARED LIBRARY CAN HAVE ONE, so only a shared library is read. + // + // This parsed every artifact, including executables, which have no + // `DT_SONAME` by construction -- an ELF that is not a shared object cannot + // carry one. On a test suite that is the entire cost of the loop spent to + // append nothing: 108 executables, 2.4 GB, one empty vector. + // + // The kind comes from `plan.linkUnits` rather than from the file, because + // that is the answer the plan already computed and reading it back out of + // the ELF is the same parse this avoids. + std::vector sharedOutputs; + for (auto const& unit : plan.linkUnits) { + if (unit.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + sharedOutputs.push_back(plan.outputDir / unit.output); + } std::vector produced; for (auto const& [artifact, stamp] : artifacts) { + if (std::ranges::find(sharedOutputs, artifact) == sharedOutputs.end()) + continue; auto facts = mcpp::platform::elf::inspect_elf_runtime(artifact); if (facts && !facts->soname.empty()) produced.push_back(facts->soname); } diff --git a/src/build/schedule/policy.cppm b/src/build/schedule/policy.cppm index edaf15b17..428080c55 100644 --- a/src/build/schedule/policy.cppm +++ b/src/build/schedule/policy.cppm @@ -124,9 +124,26 @@ std::string requested_switch(const manifest::Manifest& m, // How many compilers this machine should run at once. // -// Precedence: MCPP_JOBS (where `--jobs` lands)> `[build] jobs`> 0, meaning -// "say nothing" and leave the backend's own default. The default is unchanged -// on purpose: altering everyone's concurrency is a behaviour change. +// Precedence: MCPP_JOBS (where `--jobs` lands)> `[build] jobs`> +// `globalDefault` (the per-machine config's `[build] default_jobs`)> 0, +// meaning "say nothing" and leave the backend's own default. The default is +// unchanged on purpose: altering everyone's concurrency is a behaviour change, +// and `globalDefault` is 0 for everyone who has not written the key. +// +// THE GLOBAL VALUE ARRIVES AS A PARAMETER, not as a config import. This +// function depends on the manifest and on the host, and nothing else; reaching +// into `$MCPP_HOME/config.toml` from here would give the schedule a second +// source of truth to keep consistent. The caller reads the config it has +// already loaded. +// +// It sits BELOW the manifest and ABOVE the backend because of what each of the +// three describes. `MCPP_JOBS` is this invocation. `[build] jobs` is this +// project, and a project that states a number has a reason the machine cannot +// know. `default_jobs` is the machine, and it is the only one of the three that +// can hold a machine fact: `--jobs` must be repeated on every invocation, and +// `[build] jobs` is per-package while `[workspace.build]` rightly refuses it, +// so a seven-member workspace would otherwise carry the number seven times and +// commit a property of one developer's laptop to the repository. // // `auto` is resolved HERE, against the machine doing the build, never frozen // into a manifest. Measured on this repository: the cold self-build takes 81.0s @@ -138,7 +155,8 @@ std::string requested_switch(const manifest::Manifest& m, // `onInvalid` is called with the offending text instead of warning directly, so // this stays free of any UI dependency and remains testable. int resolve_jobs(const manifest::Manifest& m, - const std::function& onInvalid = {}); + const std::function& onInvalid = {}, + int globalDefault = 0); // `requested` is the user's switch: "auto" (default), "on", "off". `hostJobs` is // the already-resolved parallelism (`--jobs` or `[build] jobs`), or 0 meaning @@ -240,7 +258,8 @@ Decision decide(const toolchain::Toolchain& tc, std::string_view requested, int } int resolve_jobs(const manifest::Manifest& m, - const std::function& onInvalid) { + const std::function& onInvalid, + int globalDefault) { auto from_text = [&](std::string_view v) -> std::optional { if (v.empty()) return std::nullopt; if (v == "auto") { @@ -261,6 +280,12 @@ int resolve_jobs(const manifest::Manifest& m, if (const char* e = std::getenv("MCPP_JOBS")) if (auto n = from_text(e)) return *n; if (auto n = from_text(m.buildConfig.jobs)) return *n; + // Not through `from_text`: this value has already been parsed as an + // integer by the config loader, so "auto" and a malformed spelling are not + // reachable here and a second diagnostic for them would describe a state + // that cannot occur. A non-positive value reads as absent, which is what + // the generated template's `0` means. + if (globalDefault> 0) return globalDefault; return 0; } diff --git a/src/cli.cppm b/src/cli.cppm index a3e4db36a..09f61c744 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -581,8 +581,14 @@ int run(int argc, char** argv) { .help("system | vendored (default) | self-contained | static")) .option(cl::Option("target").takes_value().multiple() .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) + // "plus any format the resolved graph provides", because the + // values are no longer a fixed list. A package declares one with + // `mcpp::provides_pack_format` and `--format ` dispatches to + // it; an unknown value names what IS available rather than a + // constant, so the help text does not have to enumerate them. .option(cl::Option("format").takes_value() - .help("tar (default; .zip for a Windows target) | dir")) + .help("tar (default; .zip for a Windows target) | dir | any " + "format the resolved graph provides (e.g. appimage, msi)")) .option(cl::Option("output").short_name('o').takes_value() .help("Override output path")) // Packaging builds RELEASE by default — the artifact leaves this @@ -659,7 +665,7 @@ int run(int argc, char** argv) { // With --target the family may be omitted entirely (taken from // the target's convention pin): // mcpp toolchain install --target x86_64-windows-gnu - .arg(cl::Arg("compiler").help("gcc | llvm | msvc (or gcc@16.1.0; legacy aliases accepted)")) + .arg(cl::Arg("compiler").help("gcc | llvm | msvc | emsdk | android-ndk (or gcc@16.1.0; legacy aliases accepted)")) .arg(cl::Arg("version").help("e.g. 16.1.0, 15, 15.1")) .option(cl::Option("target").takes_value().help( "Install the toolchain payload for
(e.g. x86_64-windows-gnu)"))) diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index d6da52da7..216232338 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -47,13 +47,25 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { opts.mode = *m; modeFromUser = true; } + // THE VALUE IS NOT VALIDATED HERE, AND THAT IS THE CHANGE. + // + // `tar` and `dir` are the archive shapes the engine owns. Everything else + // is a name a package provides, and which names those are is a property of + // the RESOLVED GRAPH -- so a refusal written here could only compare + // against a constant, which is exactly the coupling this whole mechanism + // exists to remove. The refusal moves to `build_and_pack`, after build + // programs have declared what they provide and before anything is + // compiled, where it can name what IS available instead of a fixed list. if (auto v = parsed.value("format")) { if (*v == "tar") opts.format = mcpp::pack::Format::Tar; else if (*v == "dir") opts.format = mcpp::pack::Format::Dir; - else { - mcpp::ui::error(std::format( - "invalid --format '{}'; expected tar | dir", *v)); + else if (v->empty()) { + mcpp::ui::error("--format needs a value: tar | dir | a format the " + "resolved graph provides"); return 2; + } else { + opts.format = mcpp::pack::Format::Dispatched; + opts.formatName = *v; } } if (auto v = parsed.value("output")) opts.output = *v; @@ -85,6 +97,23 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { auto route = mcpp::pack::route_pack_target(parsed.positional(0)); if (!route) { mcpp::ui::error(route.error()); return 2; } if (route->library) { + // A DISPATCHED FORMAT IS AN APPLICATION-BUNDLE OUTPUT, and a library + // package has no bundle: `mcpp pack ` produces an interface plus + // prebuilt binaries for one or more triples, and there is no single + // staged tree for a member to turn into an installer. Refused rather + // than ignored, because ignoring it would report `Packed` and hand back + // a library package while the user asked for an installer. + if (opts.format == mcpp::pack::Format::Dispatched) { + mcpp::ui::error(std::format( + "--format {} is a distributable produced from a program's staged " + "bundle, and '{}' is a library target.\n" + " A library package ships an interface plus prebuilt binaries " + "per triple; there is no\n" + " single staged tree to hand a distribution format.\n" + " use: --format tar | dir, or name a program target", + opts.formatName, route->targetName)); + return 2; + } if (modeFromUser) { mcpp::ui::warning(std::format( "--mode is an application-bundle depth and does not apply to the " diff --git a/src/config.cppm b/src/config.cppm index 75d4b1ae3..08a1f5ef4 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -94,7 +94,6 @@ struct GlobalConfig { // From config.toml [build] std::int64_t defaultJobs = 0; - std::string defaultBackend = "ninja"; // From config.toml [toolchain] (M5.5) // default = "@" e.g. "gcc@15.1.0" @@ -355,7 +354,6 @@ search_ttl_seconds = 3600 [build] default_jobs = 0 -default_backend = "ninja" )"; write_file(path, tmpl); return std::filesystem::exists(path); @@ -515,7 +513,6 @@ std::expected load_or_init( cfg.indexAutoRefresh = doc->get_bool("index.auto_refresh").value_or(true); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); - cfg.defaultBackend = doc->get_string("build.default_backend").value_or("ninja"); cfg.defaultToolchain = doc->get_string("toolchain.default").value_or(""); cfg.defaultTarget = doc->get_string("toolchain.default_target").value_or(""); diff --git a/src/doctor.cppm b/src/doctor.cppm index e393cf408..4268d5620 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -440,7 +440,7 @@ export int doctor_report() { // in `toolchain list`; doctor kept its own copy, so the // two commands disagreed about the same machine. auto bin = mcpp::toolchain::payload_frontend( - vEntry.path(), mcpp::toolchain::to_xim_package(s), s.family); + vEntry.path(), mcpp::toolchain::to_xim_package(s)); if (bin.empty()) continue; sawAny = true; diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index d885e3a75..62169f998 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -223,55 +223,116 @@ bool is_well_formed_module_name(std::string_view name) { return is_dotted(name.substr(0, colon)) && is_dotted(name.substr(colon + 1)); } -// Strip a trailing line comment ("//..."). -std::string_view strip_line_comment(std::string_view s) { - auto p = s.find("//"); - if (p == std::string_view::npos) return s; - return s.substr(0, p); -} - -// Remove C++ raw-string-literal bodies from a line, tracking multi-line raw -// strings across calls via (in_raw, raw_close). Returns the code-only portion -// with raw-string contents blanked out. +// Blank out everything on a line that is not code: block-comment bodies, line +// comment tails, and raw-string bodies. Removed characters become spaces, so a +// column reported against the result still points at the source. +// +// ONE PASS OVER THREE STATES, and that is the whole point of the function. +// Code, block comment and raw string are mutually exclusive and decided by +// whichever opener comes first, which no fixed order of separate passes can +// express. The scanner had two passes -- raw strings, then an unconditional +// `find("//")` -- and no block-comment state at all, which produced a wrong +// answer in BOTH directions (measured on 2026年9月10日.2): +// +// /* the trimmed line IS `module (exe)`, so the matcher +// module (exe) fired inside a comment. With a well-formed name +// */ (`export module y;`) it was worse than an error: +// the graph recorded a plain .cpp as the producer of +// `y.gcm`, promised a BMI the compiler never writes, +// and a real importer of `y` was then told `imports +// must be built before being imported`. +// +// // R"( the raw-string pass ran first and did not know the +// import x; opener was commented out, so it entered raw mode +// and blanked every following line until a `)"` that +// never comes. `import x;` was invisible to the +// scanner and visible to the compiler -- a MISSING +// DEPENDENCY EDGE, which is a build-order race rather +// than a deterministic refusal, and therefore worse. // -// Without this, a template that embeds source text — e.g. the `mcpp new -// --template gui` skeleton stored as R"GUI( ... import imgui.core; ... )GUI" -// in scaffold/create.cppm — has its inner `import` lines misdetected as real -// module imports, producing spurious "imported but not provided" warnings. -// Ordinary "..." strings are intentionally left as-is: the import/module -// matcher only fires on lines whose trimmed text *starts with* the keyword, -// which a string body can only do when it spans lines (i.e. a raw string). -std::string strip_raw_strings(std::string_view line, bool& in_raw, - std::string& raw_close) { - std::string out; +// /* */ import x; the matcher only fires on a line whose trimmed text +// starts with the keyword, and this one starts with +// `/*`. Also missed. +// +// The comment that admitted all of this is worth quoting, because the argument +// was sound and its enumeration was short by one: "Ordinary "..." strings are +// intentionally left as-is: the import/module matcher only fires on lines whose +// trimmed text *starts with* the keyword, which a string body can only do when +// it spans lines (i.e. a raw string)." A block comment can do it too. +// +// NOT A LEXER. Ordinary string and character literals are skipped over rather +// than tokenised -- their contents are left in place, because the only question +// asked of the result is whether the trimmed line starts with a keyword and a +// `"..."` body cannot begin a line with one. Skipping them is nonetheless +// required, so that `const char* s = "a /* b";` does not open a comment that +// swallows the rest of the file. Nesting is not stripped because `/*` does not +// nest in C++. +std::string strip_noncode(std::string_view line, bool& in_block, bool& in_raw, + std::string& raw_close) { + std::string out(line); + const std::size_t n = out.size(); + auto blank = [&](std::size_t from, std::size_t to) { + for (std::size_t k = from; k < to && k < n; ++k) out[k] = ' '; + }; + std::size_t i = 0; - while (i < line.size()) { + while (i < n) { if (in_raw) { auto p = line.find(raw_close, i); - if (p == std::string_view::npos) return out; // rest of line is raw body + if (p == std::string_view::npos) { blank(i, n); return out; } + blank(i, p + raw_close.size()); i = p + raw_close.size(); in_raw = false; raw_close.clear(); continue; } + if (in_block) { + auto p = line.find("*/", i); + if (p == std::string_view::npos) { blank(i, n); return out; } + blank(i, p + 2); + i = p + 2; + in_block = false; + continue; + } + if (line[i] == '/' && i + 1 < n && line[i + 1] == '/') { + blank(i, n); + return out; + } + if (line[i] == '/' && i + 1 < n && line[i + 1] == '*') { + in_block = true; + blank(i, i + 2); + i += 2; + continue; + } // Raw-string opener: R"delim( ... )delim" (delim is up to 16 chars, // no '(' / whitespace per the standard). Optional u8/u/U/L prefixes // precede the R; we only need to spot the R" boundary. - if (line[i] == 'R' && i + 1 < line.size() && line[i + 1] == '"') { + if (line[i] == 'R' && i + 1 < n && line[i + 1] == '"') { std::size_t d = i + 2; std::string delim; - while (d < line.size() && line[d] != '(' && (d - (i + 2)) < 16) { + while (d < n && line[d] != '(' && (d - (i + 2)) < 16) { delim.push_back(line[d]); ++d; } - if (d < line.size() && line[d] == '(') { + if (d < n && line[d] == '(') { raw_close = ")" + delim + "\""; in_raw = true; + blank(i, d + 1); i = d + 1; continue; } } - out.push_back(line[i]); + if (line[i] == '"' || line[i] == '\'') { + const char q = line[i]; + std::size_t k = i + 1; + while (k < n) { + if (line[k] == '\\') { k += 2; continue; } + if (line[k] == q) { ++k; break; } + ++k; + } + i = k; // contents kept; only the scan position moves past them + continue; + } ++i; } return out; @@ -741,14 +802,15 @@ std::expected scan_file(const std::filesystem::path& file std::size_t lineno = 0; bool in_raw = false; // inside a multi-line raw string std::string raw_close; // active )delim" terminator + bool in_block = false; // inside a /* ... */ block comment std::string line; while (std::getline(is, line)) { ++lineno; - // Blank out raw-string-literal bodies first so embedded source text - // (e.g. scaffold templates) isn't misparsed as imports. - std::string code = strip_raw_strings(line, in_raw, raw_close); - std::string_view sv = strip_line_comment(code); - sv = trim(sv); + // Comment bodies, comment tails and raw-string bodies are not code. + // One pass, because the three states are decided by whichever opener + // comes first -- see strip_noncode. + std::string code = strip_noncode(line, in_block, in_raw, raw_close); + std::string_view sv = trim(code); if (sv.empty()) continue; // Track preprocessor depth (we only need to know if we're inside #if). diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 413800c53..b2437b1f8 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -50,11 +50,33 @@ export namespace mcpp::pack { enum class Mode { None, Static, BundleProject, BundleAll }; -enum class Format { Tar, Dir }; +// WHAT SHAPE THE OUTPUT TAKES, and the third value is the one that is not a +// shape the engine knows. +// +// `tar` and `dir` answer the same question `msi` and `appimage` answer, so they +// belong on one axis -- which is why this is a wider set of values for one flag +// rather than a second flag. `Dispatched` carries a name the engine has never +// heard: `mcpp pack --format appimage` finds the provider among the resolved +// dependencies and hands it the staged tree, exactly as `--target` reaches a +// triple the engine did not have to know individually. +// +// The engine keeps `Tar` and `Dir` because an archive that extracts and runs is +// universal in the only sense that matters here: it needs no knowledge of +// anyone else's release. dpkg's control fields, AppImage's runtime, WiX's +// schema and Apple's notarisation each couple an mcpp release to a release mcpp +// does not control. +enum class Format { Tar, Dir, Dispatched }; struct Options { Mode mode = Mode::BundleProject; Format format = Format::Tar; + // The `--format` value when `format == Dispatched`. Empty otherwise. + // + // Not validated by the CLI, and deliberately: the set of valid values is a + // property of the RESOLVED GRAPH, so the refusal has to wait until build + // programs have declared what they provide. It arrives before anything is + // compiled, which is the earliest point at which it can be exact. + std::string formatName; std::filesystem::path output; // empty = derive from manifest std::string targetTriple; // empty = host // Where a dependency NAME may be resolved to a file. @@ -1149,6 +1171,29 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // it and nothing has modified either one at this point (patchelf runs // further down). What changes is only which directory `$ORIGIN` // expands to while the loader is looking. + // + // A NON-ELF ARTIFACT REACHING THIS POINT ASSUMED ELF BY EXCLUSION. + // PE and Mach-O are refused by name above `run()`'s `#else`; nothing + // between there and here asks what is LEFT actually is ELF, because + // ELF used to be the only format left once those two were excluded. + // wasm32-emscripten is the first target where that assumption is + // false: `binfmt::identify` reports `Format::Unknown` for a `.wasm` + // module (it carries none of the three magics), and `ldd_parse` + // below runs the file through the same LD_TRACE_LOADED_OBJECTS + // mechanism the Mach-O branch above refuses by name rather than + // risk — on a host where `.wasm` is registered in `binfmt_misc`, + // that does not fail, it RUNS the module. + if (auto fmt = mcpp::pack::binfmt::identify(plan.builtBinary).format; + fmt != mcpp::pack::binfmt::Format::Elf) { + return std::unexpected(Error{std::format( + "cannot package the {} artifact '{}' yet.\n" + " Its dependency closure is resolved by running the " + "artifact under its own\n" + " dynamic linker, and this file is neither ELF, PE nor " + "Mach-O -- there is no\n" + " such linker to ask.", + mcpp::pack::binfmt::format_name(fmt), plan.binaryName)}); + } auto deps = ldd_parse(plan.builtBinary); if (!deps) return std::unexpected(Error{std::format( "ldd failed on {}: {}", plan.builtBinary.string(), deps.error())}); diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index bebbe7629..8d1ce9b79 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -17,7 +17,9 @@ import mcpp.build.ninja; import mcpp.build.plan; import mcpp.config; import mcpp.fetcher.progress; +import mcpp.manifest; import mcpp.pack; +import mcpp.pack.stage_tree; import mcpp.pack.strip; import mcpp.toolchain.model; import mcpp.toolchain.registry; @@ -56,8 +58,24 @@ export int build_and_pack(Options opts, bool modeFromUser, ov.profile = opts.profile; ov.profile_fallback = "release"; + // QUIET FOR A DISPATCHED FORMAT, AND ONLY UNTIL THE VALUE IS VALIDATED. + // + // `mcpp pack --format bogus` must write NOTHING to stdout and exit 2 -- + // the machine-output contract, asserted by + // tests/e2e/202_machine_output_contract.sh, because this is the path a + // client hits when it probes an mcpp for a capability. The set of valid + // values is a property of the resolved graph, so the refusal cannot be + // decided until prepare has run, and prepare narrates what it resolves. + // + // Nothing is lost when the value IS valid: the dispatch pass prepares a + // second time and prints the same lines, so a successful + // `pack --format ` narrates once rather than twice. + const bool quietUntilValidated = + opts.format == mcpp::pack::Format::Dispatched && !mcpp::ui::is_quiet(); + if (quietUntilValidated) mcpp::ui::set_quiet(true); auto ctx = mcpp::build::prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, /*extraTargets=*/{}, ov); + if (quietUntilValidated) mcpp::ui::set_quiet(false); if (!ctx) { mcpp::ui::error(ctx.error()); return 2; @@ -83,15 +101,70 @@ export int build_and_pack(Options opts, bool modeFromUser, && opts.targetTriple.empty() && ctx->tc.targetTriple.find("-musl") == std::string::npos) { // Need to re-prepare the build with the musl target. - mcpp::build::BuildOverrides ov2; - ov2.target_triple = "x86_64-linux-musl"; - ov2.profile = opts.profile; - ov2.profile_fallback = "release"; - auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov2); + // + // `ov` IS MUTATED RATHER THAN SHADOWED. It has to stay the record of + // what produced `ctx`, because the dispatch pass below re-enters + // prepare with the same overrides plus two fields -- and a second + // overrides object left behind here would make that pass differ from + // this build in a way nothing states. + ov.target_triple = "x86_64-linux-musl"; + // Quiet on the same grounds as the first prepare: this one also runs + // before `--format` has been validated. + if (quietUntilValidated) mcpp::ui::set_quiet(true); + auto ctx2 = mcpp::build::prepare_build(false, false, {}, ov); + if (quietUntilValidated) mcpp::ui::set_quiet(false); if (!ctx2) { mcpp::ui::error(ctx2.error()); return 2; } ctx = std::move(ctx2); } + // ─── Is the requested format one anything provides? ────────────── + // + // BEFORE THE BUILD, because a refusal that arrives after a full compile is + // a worse refusal, and because this is the earliest point at which it can + // be exact: build programs have now run and declared what they provide. + // + // The set is read from a pass that asked for NOTHING. That is what the + // "declare unconditionally, submit conditionally" rule buys -- a member + // that declared only when asked would leave this list empty exactly when a + // user names a format, and the refusal would name nothing. + if (opts.format == mcpp::pack::Format::Dispatched) { + auto const& provided = ctx->plan.providedPackFormats; + if (std::ranges::find(provided, opts.formatName) == provided.end()) { + std::string avail; + for (auto b : mcpp::pack::kBuiltinPackFormats) + avail += (avail.empty() ? "" : ", ") + std::string(b); + for (auto const& f : provided) { + if (mcpp::pack::is_builtin_pack_format(f)) continue; + avail += ", " + f; + } + mcpp::ui::error(std::format( + "unknown --format '{}'.\n" + " available in this build: {}\n" + " A format past `tar` and `dir` comes from a package in the " + "resolved graph, which declares\n" + " it with `mcpp::provides_pack_format(\"\")` in its build " + "program. Add the package\n" + " that provides '{}' to [build-dependencies] and activate its " + "feature.", + opts.formatName, avail, opts.formatName)); + return 2; + } + } + + // A package claiming a built-in name is silently unreachable, since the + // parser resolves `tar` and `dir` before consulting the graph at all. + // + // OUTSIDE THE DISPATCH BRANCH ABOVE, because the mistake is in the PACKAGE + // and does not depend on what this invocation asked for. Reported on every + // pack, so the author hears it on the plain `mcpp pack` they are most + // likely to run. + for (auto const& f : ctx->plan.providedPackFormats) + if (mcpp::pack::is_builtin_pack_format(f)) + mcpp::ui::warning(std::format( + "a package in this graph declares `mcpp:pack-format={}`, which " + "is one of the archive shapes `mcpp pack` owns; `--format {}` " + "will always select the built-in and never that package", f, f)); + auto be = mcpp::build::make_ninja_backend(); mcpp::build::BuildOptions bo; auto br = be->build(ctx->plan, bo); @@ -213,13 +286,177 @@ export int build_and_pack(Options opts, bool modeFromUser, mcpp::pack::mode_cli_name(plan->opts.mode), plan->strip ? ", stripped" : "")); - auto r = mcpp::pack::run(*plan, *cfg); - if (!r) { - mcpp::ui::error(r.error().message); - return 1; + // STAGING IS A SERVICE TO THE PROVIDER, NOT A PRECONDITION FOR DISPATCH. + // + // For `--format tar` and `--format dir` the staged tree IS the product, so + // a staging failure is the command failing. For a DISPATCHED format it is + // an input the provider may or may not want, and treating it as a + // precondition made every dispatched format unreachable on any target + // whose built-in bundling is refused. + // + // Measured on macos-15 with mcpp 2026年9月11日.1: `mcpp pack --format app` + // never reached the dispatch, because `pack::run` refuses a Mach-O PROGRAM + // outright -- the built-in closure walk is `LD_TRACE_LOADED_OBJECTS`, which + // is glibc's, and dyld ignores it and runs the program instead. That + // refusal is correct about the built-in archive and says nothing about + // whether a `.app` bundler can work, since a bundler that names one + // program needs no closure walk at all. The engine was answering a + // question the provider had not been asked. + // + // So the failure is REPORTED AND CARRIED rather than swallowed: the reason + // is printed as a warning, `pack_stage_dir` stays empty, and + // `${mcpp.stage_dir}` then refuses at expansion naming that reason. A + // provider that reads the tree gets a precise diagnostic; one that does not + // proceeds. Nothing is silently degraded -- what changes is who decides. + std::string stageFailure; + if (auto r = mcpp::pack::run(*plan, *cfg); !r) { + if (opts.format != mcpp::pack::Format::Dispatched) { + mcpp::ui::error(r.error().message); + return 1; + } + stageFailure = r.error().message; + mcpp::ui::warning(std::format( + "no staged tree for --format {}: {}\n" + " A format that consumes ${{mcpp.stage_dir}} cannot be produced " + "here; one that names a\n" + " built file with ${{mcpp.target_file:}} is unaffected.", + opts.formatName, stageFailure)); } + // The staged tree is now on disk and final -- past the closure, the + // `$ORIGIN` rewriting, the strip and the debug split. Describe it, so an + // action that consumes it has something whose CONTENT changes when the + // staged set does. Best-effort: see write_stage_manifest. Skipped when + // staging did not happen, so no manifest describes a tree that is not + // there. + if (stageFailure.empty()) mcpp::pack::write_stage_manifest(plan->stagingRoot); + auto pathCtx = mcpp::fetcher::make_path_ctx(&*cfg, ctx->projectRoot); + + // ─── The dispatch pass ─────────────────────────────────────────── + // + // A `role = "artifact"` action is a ninja edge, and the staged tree is + // produced here, in C++, AFTER ninja has finished. So an artifact action + // cannot depend on the staged tree in the pass that built it, and a + // single-pass `--format ` is not expressible. Two passes are, and + // every value this one needs was answered by the first: + // + // `ov` the overrides that produced the build above + // `plan->stagingRoot` from make_plan, which resolved the triple + // `opts.formatName` the request, already checked against the graph + // + // NOTHING IS RE-DERIVED, and that is the whole discipline of this block. + // `stagingRoot` is a function of the package name, the version, the + // resolved triple and the mode; the resolved triple is not known until a + // prepare has run, so computing it a second time before prepare -- from the + // host triple, say -- is the shape where two derivations of one value agree + // on every machine the author has and disagree on one they do not. + if (opts.format == mcpp::pack::Format::Dispatched) { + // WHICH ARTIFACT ACTIONS THIS BUILD ALREADY HAD, before a format was + // requested. The dispatch below reports what the REQUEST introduced, + // and this is the other half of that subtraction. + std::set> preexistingArtifacts; + for (auto const& a : ctx->plan.actions) + if (a.role == mcpp::manifest::BuildAction::Role::Artifact) + preexistingArtifacts.emplace(a.packageName, a.id); + + ov.pack_format = opts.formatName; + // Empty when staging was refused, which is what makes + // `${mcpp.stage_dir}` refuse with the reason attached rather than + // expand to a directory that does not exist. + ov.pack_stage_dir = stageFailure.empty() ? plan->stagingRoot + : std::filesystem::path{}; + ov.pack_stage_reason = stageFailure; + auto distCtx = mcpp::build::prepare_build(false, false, {}, ov); + if (!distCtx) { mcpp::ui::error(distCtx.error()); return 2; } + + // WHICH ACTIONS ARE THE DISTRIBUTABLE: the artifact actions the REQUEST + // INTRODUCED. An action present in both passes existed before anyone + // asked for a format -- a codesign stamp, a size budget -- and + // reporting one as the package would be a wrong answer that looks like + // a right one. + // + // THE FIRST VERSION ASKED A NARROWER QUESTION AND GOT IT WRONG. It + // collected only actions naming `${mcpp.stage_dir}`, on the assumption + // that a distributable consumes the staged closure. Not every format + // does: an `.msi` built from ONE named program takes + // `${mcpp.target_file:}` and never looks at the tree, which is + // the shape section 6 of the design record recommends -- "name the + // input, do not harvest a directory", after a bind path that resolved + // to nothing produced a valid, empty, 52 KB installer. So the member + // that followed the guidance was the member the check refused, and the + // workaround was to name the placeholder as an unused input purely to + // satisfy it. Presence-in-this-pass is the property actually wanted, + // and it needs nothing of the member. + // + // Identity is (package, id): an id is unique within the package that + // declared it and nothing more. + std::vector distOutputs; + for (auto const& a : distCtx->plan.actions) { + if (a.role != mcpp::manifest::BuildAction::Role::Artifact) continue; + if (preexistingArtifacts.contains({a.packageName, a.id})) continue; + for (auto const& o : a.outputs) distOutputs.push_back(o); + } + // DECLARED AND THEN SUBMITTED NOTHING. The half of the contract a + // member is most likely to get wrong is the gate, and a member whose + // gate never opens leaves a pass that succeeds and produces no + // package. Refused by name rather than reported as success. + if (distOutputs.empty()) { + mcpp::ui::error(std::format( + "no action claimed --format '{}'.\n" + " A package declared it provides this format, and no build " + "program submitted a new\n" + " `role = \"artifact\"` action when it was asked for.\n" + " The provider must gate on the request and not on anything " + "else:\n" + " mcpp::provides_pack_format(\"{}\"); " + "// always\n" + " if (std::string_view(mcpp::pack_format()) == \"{}\") ..." + " // then submit", + opts.formatName, opts.formatName, opts.formatName)); + return 1; + } + + mcpp::ui::info("Distributing", std::format("{} v{} (--format {})", + plan->packageName, plan->packageVersion, opts.formatName)); + + // NO EXPLICIT GOALS. Everything but the dist edges is already up to + // date from the build above, so a full drive costs a graph scan and + // nothing else -- and an explicit goal set is how the 0.0.104 soname + // aliases went missing, because an edge reachable only through + // `default` is skipped under one. + mcpp::build::BuildOptions dbo; + auto dr = be->build(distCtx->plan, dbo); + if (!dr) { + if (!dr.error().diagnosticOutput.empty()) { + std::fputs(dr.error().diagnosticOutput.c_str(), stderr); + if (dr.error().diagnosticOutput.back() != '\n') std::fputs("\n", stderr); + } + mcpp::ui::error(dr.error().message); + return 1; + } + + // THE CRITERION IS THE FILE, NOT THE EXIT CODE. A cached build program + // replaying the first pass's answer, or a tool that writes nothing and + // exits 0, both leave ninja reporting success -- and section 2's + // measured failure was a packaging step that succeeded while carrying + // nothing. + std::error_code ec; + for (auto const& o : distOutputs) { + auto abs = std::filesystem::path(o).is_absolute() + ? std::filesystem::path(o) : distCtx->plan.outputDir / o; + if (!std::filesystem::is_regular_file(abs, ec) + && !std::filesystem::is_directory(abs, ec)) { + mcpp::ui::error(std::format( + "--format {} reported success and produced nothing at {}", + opts.formatName, abs.string())); + return 1; + } + mcpp::ui::status("Packed", mcpp::ui::shorten_path(abs, pathCtx)); + } + return 0; + } + auto outPath = (opts.format == mcpp::pack::Format::Tar) ? plan->archivePath : plan->stagingRoot; mcpp::ui::status("Packed", mcpp::ui::shorten_path(outPath, pathCtx)); diff --git a/src/pack/stage_tree.cppm b/src/pack/stage_tree.cppm new file mode 100644 index 000000000..b24f25241 --- /dev/null +++ b/src/pack/stage_tree.cppm @@ -0,0 +1,125 @@ +// mcpp.pack.stage_tree — what a staged tree promises an artifact action, and +// the one file that says so. +// +// `mcpp pack` has always computed a staged tree: the dependency closure after +// the strip policy, the debug-symbol split and `include`/`exclude`. Until +// `${mcpp.stage_dir}` it then compressed the tree and the directory was gone, +// so a `.deb`, an AppImage, a `.app` and an `.msi` each had to rebuild the same +// closure. Exposing it is one addition that serves every format and encodes no +// format's knowledge, which is the test for whether something belongs in the +// engine at all. +// +// WHY THERE IS A MANIFEST FILE AND NOT JUST A DIRECTORY. ninja identifies an +// input by a path and compares an mtime. A directory's mtime moves when its +// immediate entries change and not when a file two levels down is replaced, so +// naming the directory as an input would make the dist edge dirty for the wrong +// reasons and clean for the wrong reasons. The manifest is CONTENT-BEARING — +// one ` ` line per staged file, sorted — so it changes +// exactly when the staged set or any staged file's length changes, and it is a +// single ordinary file that ninja can compare. +// +// IT IS A SIBLING OF THE TREE, NOT A MEMBER OF IT. A file inside the staged +// directory would be collected by every format that packages the directory +// wholesale, and would then ship inside the user's installer. The sibling +// spelling is what keeps a mechanism the engine added from appearing in a +// product it does not own. +// +// The manifest deliberately records SIZES rather than content hashes. The tree +// can be hundreds of megabytes and is rebuilt on every pack; hashing it would +// make the common case pay for a distinction the uncommon case does not need, +// because a staged file whose length is unchanged and whose bytes differ can +// only have come from a rebuild, and a rebuild moved the link output that the +// dist edge also depends on. + +module; +#include + +export module mcpp.pack.stage_tree; + +import std; + +export namespace mcpp::pack { + +// The manifest that describes the tree staged at `stagingRoot`. +// +// A SIBLING, DERIVED FROM THE NAME RATHER THAN PLACED INSIDE. Spelled in one +// function because two readers need the same answer for different reasons: +// `prepare` names it as an implicit input while the file does not yet exist, +// and `pack::run` writes it. A second derivation of a path is a path that +// disagrees on the platform whose separator the author did not test. +std::filesystem::path stage_manifest_path(const std::filesystem::path& stagingRoot) { + auto p = stagingRoot; + p += ".stage-manifest"; + return p; +} + +// Write the manifest for the tree now on disk at `stagingRoot`. +// +// Best-effort by construction and deliberately so: the manifest is a +// dependency-tracking convenience, and a pack that produced a correct tree must +// not fail because a sibling bookkeeping file could not be written. A missing +// manifest makes the dist edge fail with ninja's own "missing and no known rule +// to make it", which names the file — a legible failure rather than a silent +// staleness. +bool write_stage_manifest(const std::filesystem::path& stagingRoot) { + std::error_code ec; + if (!std::filesystem::is_directory(stagingRoot, ec)) return false; + + std::vector lines; + for (auto const& entry : + std::filesystem::recursive_directory_iterator( + stagingRoot, std::filesystem::directory_options::skip_permission_denied, ec)) + { + if (ec) break; + // Symlinks are recorded by NAME AND NOT FOLLOWED. `bundle-all` + // dereferences a soname link while staging, so what remains is a real + // file; a link that survives points outside the tree, and following it + // would make the manifest describe a file the package does not carry. + if (!entry.is_regular_file(ec) || entry.is_symlink()) { + if (entry.is_symlink()) + lines.push_back(std::format("link {}", + std::filesystem::relative(entry.path(), stagingRoot, ec).generic_string())); + continue; + } + auto rel = std::filesystem::relative(entry.path(), stagingRoot, ec).generic_string(); + if (ec || rel.empty()) continue; + lines.push_back(std::format("{} {}", + std::filesystem::file_size(entry.path(), ec), rel)); + } + // Sorted, because a directory iteration order is not a promise. Two packs + // of one tree must produce identical bytes or the dist edge is dirty on + // every run for no reason. + std::ranges::sort(lines); + + std::string text; + for (auto const& l : lines) { text += l; text.push_back('\n'); } + + auto out = stage_manifest_path(stagingRoot); + // Compared before writing, for the reason `mcpp.build.stage` gives at + // length: rewriting identical bytes moves the mtime, and a moved mtime on + // an input is indistinguishable from a changed input. A pack that staged + // the same tree twice would rebuild the distributable both times. + if (std::ifstream in(out, std::ios::binary); in) { + std::string old((std::istreambuf_iterator(in)), + std::istreambuf_iterator()); + if (old == text) return true; + } + std::ofstream os(out, std::ios::binary | std::ios::trunc); + if (!os) return false; + os.write(text.data(), static_cast(text.size())); + return static_cast(os); +} + +// The two names `--format` answers for without consulting the graph. +// +// Held here rather than in the CLI because two layers need the same list: the +// parser decides whether a value is a built-in or a dispatch, and the +// declaration check refuses a package that claims one of them. A package +// claiming `tar` would be silently unreachable, since the built-in wins. +constexpr std::array kBuiltinPackFormats{"tar", "dir"}; + +bool is_builtin_pack_format(std::string_view name) { + return std::ranges::find(kBuiltinPackFormats, name) != kBuiltinPackFormats.end(); +} + +} // namespace mcpp::pack diff --git a/src/pack/strip.cppm b/src/pack/strip.cppm index ab2143732..6707ebb51 100644 --- a/src/pack/strip.cppm +++ b/src/pack/strip.cppm @@ -166,7 +166,12 @@ bool debug_info_is_in_band(std::string_view canonicalTriple) { seg.push_back(canonicalTriple.substr(i, j - i)); i = j + 1; } - if (seg.size()>= 2 && seg[1] == "macos") return false; // debug map + .dSYM + // Mach-O, not "macOS": iOS carries the same debug map + out-of-band + // .dSYM as macOS (same ld64, same object format). This module takes a + // string rather than a `Triple` on purpose (see the note on + // `debug_info_is_in_band` above), so the grouping `is_mach_o()` states is + // spelled out here instead of asked of it. + if (seg.size()>= 2 && (seg[1] == "macos" || seg[1] == "ios")) return false; if (seg.size()>= 3 && seg[2] == "msvc") return false; // separate .pdb return true; } diff --git a/src/toolchain/clang.cppm b/src/toolchain/clang.cppm index c94089d6e..27b5515b6 100644 --- a/src/toolchain/clang.cppm +++ b/src/toolchain/clang.cppm @@ -127,6 +127,47 @@ std::optional find_libcxx_std_module_source( } } + // SECOND PROBE: ASK WHICH libc++ WILL BE LINKED, THEN LOOK BESIDE IT. + // + // `-print-library-module-manifest-path` is the primary answer and the one + // llvm's own payload gives. `em++` does not forward it -- it answers + // `em++: error: no input files` -- so a toolchain that ships the surface + // was reported as shipping none, and `import std` was refused on a target + // where it demonstrably works. + // + // `--print-file-name=libc++.a` IS forwarded, and what it names is exactly + // the library the link will use, so the surface beside it is the surface + // that matches. Measured 2026年09月11日: + // + // xim:llvm 22.1.8 /bin/../lib/x86_64-unknown-linux-gnu/libc++.a + // surface at /bin/../share/libc++/v1/ (3 up) + // xim:emsdk 6.0.9 /emscripten/cache/sysroot/lib/wasm32-emscripten/libc++.a + // surface at /.../sysroot/share/libc++/v1/ (2 up) + // + // THE TWO DEPTHS ARE WHY THIS WALKS RATHER THAN INDEXES. A fixed `up 3` + // would have been written against llvm, passed, and then been a guessed + // layout for the next payload -- which is the mistake the line this + // replaces made with `bin/../share`. + if (auto lib_r = mcpp::toolchain::run_capture(std::format( + "{}{} --print-file-name=libc++.a {}", + envPrefix, mcpp::xlings::shq(cxx_binary.string()), + mcpp::platform::null_redirect))) { + std::filesystem::path lib(mcpp::toolchain::trim_line(*lib_r)); + // A driver that cannot place the library echoes the bare name back. + if (lib.has_parent_path()) { + std::error_code ec; + auto dir = std::filesystem::weakly_canonical(lib.parent_path(), ec); + if (ec) dir = lib.parent_path(); + for (int up = 0; up < 4 && !dir.empty(); ++up) { + auto cand = dir / "share" / "libc++" / "v1" / "std.cppm"; + if (std::filesystem::exists(cand)) return cand; + if (!dir.has_relative_path()) break; + dir = dir.parent_path(); + } + } + } + + // THIRD: the layout guess, kept for a driver that answers neither probe. auto root = cxx_binary.parent_path().parent_path(); auto fallback = root / "share" / "libc++" / "v1" / "std.cppm"; if (std::filesystem::exists(fallback)) return fallback; @@ -157,10 +198,13 @@ void enrich_toolchain(Toolchain& tc, const std::string& envPrefix) { if (auto p = mcpp::toolchain::msvc::find_std_module_source()) { tc.stdModuleSource = *p; tc.hasImportStd = true; - // This is MSVC STL's std.ixx — the STL's own C++20 policy applies, - // not libc++'s. tc.version is clang's here, so it cannot answer the - // cl-banner question; stay strict (the STL is the binding side). - tc.importStdMinLevel = 23; + // This is MSVC STL's std.ixx, so the STL's own C++20 policy + // applies rather than libc++'s. `tc.version` is clang's and cannot + // answer it -- which is a reason to change the input, not to assume + // the worst. The toolset version is in the path of the file just + // selected, and that is the STL that will be compiled. + tc.importStdMinLevel = + mcpp::toolchain::msvc::std_module_min_level_for_stl(*p); } } #endif @@ -208,6 +252,35 @@ std::vector std_module_build_commands(const Toolchain& tc, // headers contributed; only the machine has to be restated. See // Toolchain::stdModuleTargetFlags. const std::string& codegenFlags = tc.stdModuleTargetFlags; + // AND THE PRECOMPILE NEEDS THE MACHINE TOO, FROM WHICHEVER SOURCE HAS IT. + // + // `stdModuleTargetFlags` reached only the CODEGEN command, on the reading + // that the first step needs headers and the second needs the machine. The + // first step needs both: a `--precompile` that does not say which target + // resolves the standard library's own `#include <__config>` against the + // BUILDING machine. + // + // It was invisible while exactly two kinds of toolchain existed. A payload + // whose compiler IS its target needs no flag, and a PACKAGE-provided module + // carries the target inside `stdModuleFlags` -- which is why the comment + // above insists that whoever sets that string states the target as well. A + // payload whose compiler serves several targets is a third kind, and it has + // neither: the NDK's one clang++ compiles for both Android arches and is + // told which by `--target` alone. Measured on `aarch64-linux-android`: + // + // clang++ -std=c++23 -Wno-reserved-module-identifier \ + // --precompile .../share/libc++/v1/std.cppm -o pcm.cache/std.pcm + // std.cppm:16:10: fatal error: '__config' file not found + // + // Five tokens, and the same error text this file already records from a + // Windows host in 2026-08 -- same cause, reached by a different route. + // + // The two sources are never both needed: `stdModuleFlags` is a SUPERSET of + // `stdModuleTargetFlags` when it is set at all (the producer builds the + // machine part first and appends the include part), so taking it in + // preference keeps `--target` off the command line twice. + const std::string& precompileFlags = + extraFlags.empty() ? codegenFlags : extraFlags; #if defined(_WIN32) // Windows: use absolute paths, raw binary path as first token // (cmd.exe strips leading quotes), shq for args with spaces. @@ -264,7 +337,7 @@ std::vector std_module_build_commands(const Toolchain& tc, cppStandardFlag, ixxFlags, sysrootFlag, - extraFlags, + precompileFlags, mcpp::xlings::shq(tc.stdModuleSource.string()), mcpp::xlings::shq(absBmi)), std::format( @@ -287,7 +360,7 @@ std::vector std_module_build_commands(const Toolchain& tc, mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, - extraFlags, + precompileFlags, mcpp::xlings::shq(tc.stdModuleSource.string()), mcpp::xlings::shq(relBmi)), std::format( @@ -315,10 +388,22 @@ std::optional find_libcxx_std_compat_source( const std::filesystem::path& cxx_binary, const std::string& envPrefix) { - // Same search strategy as find_libcxx_std_module_source but for std.compat - auto root = cxx_binary.parent_path().parent_path(); - auto p = root / "share" / "libc++" / "v1" / "std.compat.cppm"; - if (std::filesystem::exists(p)) return p; + // DERIVED FROM THE SIBLING, NOT SEARCHED FOR SEPARATELY. + // + // The comment here used to say "same search strategy as + // find_libcxx_std_module_source", and it was not: that function has three + // probes and this one had the last of them, so on any payload the layout + // guess does not reach -- emsdk, for one -- `std` was found and + // `std.compat` was not, from one directory. + // + // `std.compat.cppm` sits beside `std.cppm` in every libc++ layout, because + // the same install rule places both. Deriving it makes the two answers + // structurally consistent rather than two searches that can disagree, + // which is what the comment claimed all along. + if (auto std_src = find_libcxx_std_module_source(cxx_binary, envPrefix)) { + auto p = std_src->parent_path() / "std.compat.cppm"; + if (std::filesystem::exists(p)) return p; + } return std::nullopt; } @@ -362,6 +447,9 @@ std::vector std_compat_build_commands(const Toolchain& tc, // Same split as the `std` builder above: the second command compiles a BMI // and needs the machine restated, not the include paths. const std::string& codegenFlags = tc.stdModuleTargetFlags; + // Same third kind of toolchain as the `std` builder above, same reason. + const std::string& precompileFlags = + extraFlags.empty() ? codegenFlags : extraFlags; // std.compat depends on std, so we need -fmodule-file=std= // Note: the path after = must NOT be shell-quoted separately; the // entire -fmodule-file flag is a single token to the compiler. @@ -397,7 +485,7 @@ std::vector std_compat_build_commands(const Toolchain& tc, mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, - extraFlags, + precompileFlags, absStdBmi, mcpp::xlings::shq(tc.stdCompatSource.string()), mcpp::xlings::shq(absBmi)), diff --git a/src/toolchain/compat.cppm b/src/toolchain/compat.cppm index acf165e45..8bdd0b159 100644 --- a/src/toolchain/compat.cppm +++ b/src/toolchain/compat.cppm @@ -38,6 +38,11 @@ struct NormalizedSpec { std::string family; // "gcc" | "llvm" | "msvc" | "openkal-llvm" std::string version; // numeric (possibly partial), or "system"; never "-musl"-suffixed triple::Triple target; // empty = host + // WHICH PAYLOAD, when the family alone does not say. `emsdk` and + // `android-ndk` both normalise to the llvm family -- their compilers ARE + // clang -- so without this the two are indistinguishable from `xim:llvm` + // in every line mcpp prints. Empty for every other spelling. + std::string payload; // Set when a legacy spelling was rewritten; `hint` is the one-line note. bool changed = false; std::string hint; @@ -126,6 +131,33 @@ std::optional normalize_spec(std::string_view compilerIn, return out; } + // ── canonical families whose payload carries its own target ───────────── + // + // NOT ALIASES AND NOT LEGACY. `em++` and the NDK's `clang++` are clang, so + // the FAMILY is llvm and there is no fourth value to invent; what these + // spellings add is which payload answers, and for emsdk also which target + // -- the payload compiles for exactly one, so a spec that names it has + // already named the target. `with_hint` is deliberately not used: a hint + // says "this spelling is old, here is the current one", and these are the + // current ones. + if (compiler == "emsdk" || compiler == "emscripten") { + out.family = "llvm"; + out.payload = "emsdk"; + if (auto t = triple::parse("wasm32-emscripten")) out.target = *t; + if (muslVersionSuffix) return std::nullopt; + return out; + } + // The NDK serves BOTH Android arches from one payload, so it must NOT set + // a target: the arch arrives from `--target` or `[target.
]`, and + // pinning one here would make `android-ndk@` mean aarch64 to a reader + // who typed it for x86_64. + if (compiler == "android-ndk" || compiler == "ndk") { + out.family = "llvm"; + out.payload = "android-ndk"; + if (muslVersionSuffix) return std::nullopt; + return out; + } + // ── legacy spellings ───────────────────────────────────────────────────── if (compiler == "clang") { // alias family → llvm out.family = "llvm"; diff --git a/src/toolchain/hostflags.cppm b/src/toolchain/hostflags.cppm index 5ab3fcb01..2c680195f 100644 --- a/src/toolchain/hostflags.cppm +++ b/src/toolchain/hostflags.cppm @@ -150,6 +150,31 @@ std::vector host_link_tokens(const Toolchain& tc, std::vector bmi_reference_tokens(std::string_view usePrefix, const std::filesystem::path& bmi); +// The first `=` token in `argv` that no switch introduces, if any. +// +// A DEFECT THAT IS ONLY VISIBLE IN THE ASSEMBLED ARGV. `bmi_reference_tokens` +// returns MSVC's reference as a PAIR -- `/reference`, then `=` -- +// because cl.exe takes the two as separate arguments. The pair's halves are +// individually well-formed, so every check that reads one token at a time +// passes while the pair is broken. Measured: a per-token de-duplicator dropped +// the second `/reference` (already present from the bundled `mcpp` module) and +// left its partner standing alone, which cl read as a source file name: +// +// c1xx: fatal error C1083: Cannot open source file: +// 'huxerui.rules.sources=...\huxerui.rules.sources.ifc' +// +// A message that names the module and the BMI and does not name the flag, so +// it reads as a missing file rather than as a missing switch. +// +// The rule: a token that carries `=` and does not itself begin with `-` or `/` +// is an argument TO something, and the token before it must be a switch. This +// is defence in depth and not the fix -- the fix is that nothing filters the +// pair any more -- but the failure it converts is expensive to diagnose from +// cl's own words, and the check costs one pass over an argv that is already +// being built. +std::optional orphaned_reference( + const std::vector& argv); + } // namespace mcpp::toolchain namespace mcpp::toolchain { @@ -159,6 +184,57 @@ std::vector host_compile_tokens(const Toolchain& tc, const PathEscape& esc) { std::vector out; + // A TOOLCHAIN THAT SHIPS ITS OWN SYSROOT IS TOLD NOTHING. + // + // What this function emits is a target's system reconstructed onto the + // command line: libc++'s headers, glibc's, the Linux UAPI headers, the + // cfg bypass, the C-runtime prefix. Every one of those is an answer mcpp + // supplies because the payload's clang does not have one. An Emscripten or + // Android SDK does: `em++` bakes `--sysroot=/.../cache/sysroot` + // into every invocation and the NDK's clang derives its bionic sysroot + // from its own install prefix. + // + // Measured before this gate, on the std module precompile for + // `wasm32-emscripten`: + // + // em++ ... -isystem'/include' -isystem'/include' + // --precompile /share/libc++/v1/std.cppm + // /include/gnu/stubs.h:7: fatal error: + // 'gnu/stubs-32.h' file not found + // + // This host's glibc headers, handed to a wasm compile. The error names a + // missing 32-bit stub, so it reads as a broken glibc payload rather than + // as a C library that has no business being there. + // + // The cfg bypass is withheld too, and deliberately: it exists to stop + // clang reading a per-install `clang++.cfg`, while `em++` is a wrapper + // whose entire job is to supply configuration. Suppressing it would be + // suppressing the toolchain. + // + // "NOTHING" WAS ONE TOKEN TOO STRONG, AND THIS FUNCTION ALREADY SAID SO + // FURTHER DOWN. The paragraph beginning "THE TRIPLE, SAID OUT LOUD" states + // the opposite rule for the same reason -- an ordinary clang emits for the + // machine it is running on unless told otherwise -- and this early return + // stood in front of it, so the stronger claim won by position. + // + // Both are right about their own object. The SYSTEM is the payload's and + // must not be reconstructed; WHICH TARGET is still mcpp's to say, because + // one NDK serves both Android arches and nothing on the command line + // otherwise distinguishes them. Measured on `aarch64-linux-android`, with + // the std module already correct: + // + // error: AST file 'std.pcm' was compiled for the target + // 'aarch64-unknown-linux-android21' but the current translation unit + // is being compiled for target 'x86_64-unknown-linux-gnu' + // + // Two machines in one build, reported by the module loader rather than by + // either compile -- and then eight cascading "use of undeclared identifier + // 'std'" errors, which is what a reader sees first. + if (auto tt = triple::parse(tc.targetTriple); tt && tt->has_own_sysroot()) { + if (!tc.crossTargetFlag.empty()) out.push_back(tc.crossTargetFlag); + return out; + } + // MSVC carries none of this on the command line: cl.exe and link.exe find // headers and import libraries through INCLUDE / LIB, which detection // synthesizes into tc.envOverrides. Emitting the GNU shapes below would @@ -293,6 +369,26 @@ std::vector host_compile_tokens(const Toolchain& tc, return out; } +std::optional orphaned_reference( + const std::vector& argv) { + auto is_switch = [](std::string_view t) { + return !t.empty() && (t.front() == '-' || t.front() == '/'); + }; + for (std::size_t i = 0; i < argv.size(); ++i) { + std::string_view t = argv[i]; + if (is_switch(t) || t.find('=') == std::string_view::npos) continue; + // A path can contain `=`, and an input file is a legitimate bare + // token. What distinguishes a reference is that its `=` precedes any + // directory separator: `=` names a module first. + auto eq = t.find('='); + auto sep = t.find_first_of("/\\"); + if (sep != std::string_view::npos && sep < eq) continue; + if (i == 0 || !is_switch(argv[i - 1])) + return std::string(t); + } + return std::nullopt; +} + std::vector bmi_reference_tokens(std::string_view usePrefix, const std::filesystem::path& bmi) { std::string_view p = usePrefix; diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index dc7fa6277..e93421460 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -490,8 +490,7 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, // From the payload ROOT, not `root/bin`: msvc keeps cl.exe // four levels deeper, and asking for `root/bin` skipped every // installed toolset silently. - auto bin = mcpp::toolchain::payload_frontend(vEntry.path(), pkg, - id->family); + auto bin = mcpp::toolchain::payload_frontend(vEntry.path(), pkg); if (bin.empty()) continue; payloads.push_back({ *id, s.version, bin }); } @@ -623,7 +622,18 @@ export int toolchain_list(const mcpp::config::GlobalConfig& cfg, if (!info->note.empty()) tags.emplace_back(info->note); if (info->defaultStatic) tags.push_back("static"); } - if (t != hostT && (t.os != hostT.os || t.arch != hostT.arch)) + // A TARGET THIS HOST CANNOT EXECUTE. Arch and OS are the usual answer, + // and deliberately not env: an `x86_64-linux-musl` artifact is static + // and runs here, so calling it cross would be false. + // + // The env axis matters for exactly one row today. An Android artifact + // needs bionic's loader at `/system/bin/linker64`, which no ordinary + // Linux host has -- so `x86_64-linux-android` agrees with this host on + // both segments the test looked at and cannot run on it. Spelled as a + // property rather than by adding `env != env`, which would have taken + // musl with it. + if (t != hostT && (t.os != hostT.os || t.arch != hostT.arch + || t.is_android())) tags.push_back("cross"); std::string out; for (auto& tag : tags) { if (!out.empty()) out += ", "; out += tag; } @@ -1092,7 +1102,7 @@ export int toolchain_set_default(const mcpp::config::GlobalConfig& cfg, // // Same rule as everywhere else in this round: installed means usable, // not present. - if (mcpp::toolchain::payload_frontend(installDir, pkg, spec->family).empty()) { + if (mcpp::toolchain::payload_frontend(installDir, pkg).empty()) { // Before "not installed", check whether this is the retired // `msvc@` spelling — otherwise the advice is to // install a toolset that does not exist and never will. diff --git a/src/toolchain/msvc.cppm b/src/toolchain/msvc.cppm index 7afa62558..f3769b235 100644 --- a/src/toolchain/msvc.cppm +++ b/src/toolchain/msvc.cppm @@ -46,15 +46,46 @@ std::optional find_std_module_source(); // Find cl.exe (for future MSVC toolchain support). std::optional find_cl(); -// Lowest -std= level MSVC STL builds the `std` module at, for a toolchain -// whose `version` is a cl banner version ("19.44.35211"). +// Lowest -std= level MSVC STL builds the `std` module at. // // microsoft/STL#3945 ("Supporting `import std;` in C++20") was fixed by // STL#3977 (merged 2023年08月31日) — the C++20 block was a policy choice with no -// technical reason behind it. That first ships in VS 2022 17.8, i.e. cl 19.38; -// older STLs still refuse and would fail inside std.ixx, so they answer 23 and -// get an actionable diagnostic from the caller instead. This is also what keeps -// the level gate reachable: every other provider answers 20. +// technical reason behind it. That first ships in VS 2022 17.8, i.e. cl 19.38 +// and toolset 14.38; older STLs still refuse and would fail inside std.ixx, so +// they answer 23 and get an actionable diagnostic from the caller instead. This +// is also what keeps the level gate reachable: every other provider answers 20. +// +// ASKED OF THE STL, NOT OF THE COMPILER, and that distinction is the whole +// reason this takes a path. +// +// The question is a property of the standard library being compiled, and two +// different compilers reach the same `std.ixx`: cl.exe under +// `windows = "msvc@system"`, and clang targeting `*-windows-msvc` when no +// libc++ std module is present. The clang path used to hardcode 23 with a +// comment saying why it could not ask -- "tc.version is clang's here, so it +// cannot answer the cl-banner question" -- which is correct about the field +// and is an argument for changing the input rather than for assuming the +// worst. Calling the banner form from there would compare a CLANG version +// number against an MSVC threshold: clang 20.x would pass it by accident and +// clang 19.x would fail it wrongly, both by asking the wrong object. +// +// The toolset version is in the path of the module source that was already +// selected -- +// +// /VC/Tools/MSVC/14.44.35207/modules/std.ixx +// +// -- and toolset `14.` pairs with cl banner `19.`, so the existing +// `>= 38` predicate transfers unchanged. Taking it from the SELECTED file +// rather than from a fresh search matters on a machine with two installations: +// the answer must describe the STL that will actually be compiled. +// +// A path with no parseable `14.` component answers 23, which keeps the +// safety the hardcode was after without charging every modern installation for +// it. +int std_module_min_level_for_stl(const std::filesystem::path& stdModuleSource); + +// The cl-banner form, kept for a toolchain whose `version` is a cl banner +// version ("19.44.35211") and no module source has been located yet. int std_module_min_level(const Toolchain& tc); // ─── Installation records (both origins) ───────────────────────────────── @@ -931,6 +962,31 @@ std::string cl_stage_command(const Toolchain& tc, } // namespace +int std_module_min_level_for_stl(const std::filesystem::path& stdModuleSource) { + // /VC/Tools/MSVC//modules/std.ixx -> up two from the file. + if (stdModuleSource.empty()) return 23; + auto toolset = stdModuleSource.parent_path().parent_path().filename().string(); + int major = 0, minor = 0; + std::size_t i = 0; + auto read = [&](int& out) { + bool any = false; + while (i < toolset.size() && toolset[i]>= '0' && toolset[i] <= '9') { + out = out * 10 + (toolset[i] - '0'); + ++i; + any = true; + } + return any; + }; + if (!read(major)) return 23; + if (i < toolset.size() && toolset[i] == '.') ++i; + if (!read(minor)) return 23; + // 14 is the toolset major for every MSVC since VS 2015 and the only value + // this mapping is defined for. Anything else is a layout this code does + // not recognise, and a guess there would be the defect it replaces. + if (major != 14) return 23; + return minor>= 38 ? 20 : 23; +} + int std_module_min_level(const Toolchain& tc) { // Two-segment compare: cppfly::compiler_major only reads the leading // integer, which is 19 for every MSVC ever shipped. Keep that function's @@ -1038,7 +1094,12 @@ std::expected enrich_toolchain_from_cl(Toolchain& tc) { tc.hasImportStd = true; } if (tc.hasImportStd) { - tc.importStdMinLevel = std_module_min_level(tc); + // The STL, not the banner. For a real cl installation the two agree by + // construction -- `std.ixx` was found under the same toolset directory + // cl came from -- and a unit test asserts that. They separate only when + // `find_msvc_tools_dir()` and the selected module source disagree, and + // there the file that will be compiled is the correct answer. + tc.importStdMinLevel = std_module_min_level_for_stl(tc.stdModuleSource); } if (auto compat = toolsDir / "modules" / "std.compat.ixx"; std::filesystem::exists(compat, ec)) { diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index dff4b0a65..44ab9de64 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -14,6 +14,7 @@ export module mcpp.toolchain.probe; import std; import mcpp.toolchain.model; +import mcpp.toolchain.triple; import mcpp.xlings; import mcpp.platform; import mcpp.log; @@ -212,6 +213,31 @@ std::vector discover_link_runtime_dirs(const std::filesystem::path& compilerBin, std::string_view targetTriple) { std::vector dirs; + + // THE COMPILER'S OWN RUNTIME IS NOT THE ARTIFACT'S, AND FOR EVERY ROW + // BEFORE THESE TWO THE DISTINCTION DID NOT MATTER. + // + // What this returns goes on the ARTIFACT's link line as `-L` and `-rpath` + // (flags.cppm's runtime_dirs). For a payload that targets its own host + // that is right: the libstdc++ beside the compiler is the libstdc++ the + // artifact links. An SDK that cross-compiles breaks the coincidence -- + // `/lib` holds the host x86-64 libraries its own clang needs to + // RUN, and its sibling `xim:gcc-runtime` is a runtime dependency of the + // toolchain, not of a wasm module. + // + // Measured on `--target wasm32-emscripten`, after the sysroot and link + // model were already gated: + // + // wasm-ld: error: unknown file type: + // /lib64/libatomic.so + // + // and `atomic_link_flag` had found that libatomic precisely because this + // function had put its directory on the link dirs. The compiler's own + // needs are `compilerRuntimeDirs`, which this does not touch and which + // still gets them. + if (auto tt = triple::parse(targetTriple); tt && tt->has_own_sysroot()) + return dirs; + auto root = compilerBin.parent_path().parent_path(); if (!targetTriple.empty()) append_existing_unique(dirs, root / "lib" / std::string(targetTriple)); diff --git a/src/toolchain/registry.cppm b/src/toolchain/registry.cppm index 609bcc342..eb36fa4b4 100644 --- a/src/toolchain/registry.cppm +++ b/src/toolchain/registry.cppm @@ -2,7 +2,8 @@ // payload mapping. // // Identity (design §4.1–§4.3): a toolchain is `family@version` (family ∈ -// gcc | llvm | msvc), a target is a canonical Triple (triple.cppm). The two +// gcc | llvm | msvc | emsdk | android-ndk), a target is a canonical Triple +// (triple.cppm). The two // axes are orthogonal: "cross", "musl" and "mingw" are NOT names — the // variant lives in the target's env segment, and cross is the host≠target // relation. Which xim PACKAGE serves a (family, version, target, host) @@ -16,6 +17,7 @@ export module mcpp.toolchain.registry; import std; +import mcpp.libs.json; import mcpp.platform; import mcpp.xlings; import mcpp.toolchain.clang; @@ -73,8 +75,31 @@ struct ToolchainSpec { bool is_host_target() const { return target.empty(); } // "gcc@16.1.0" — the toolchain axis alone (config persistence, matching). + // WHICH PAYLOAD ANSWERED, NOT ONLY WHICH FAMILY. + // + // `emsdk@6.0.9` normalises to the llvm family, because `em++` IS clang and + // a fourth family value would be a false claim about the compiler. The + // consequence was a display line reading `Resolved llvm@6.0.9`, which is + // indistinguishable from the real `xim:llvm` and is not what the user + // typed. `mcpp toolchain list` has the same problem, and the matrix scan + // takes one toolchain per family, so two llvm-family payloads on one host + // could not both be enumerated. + // + // The family and the payload are two questions: + // + // family what flag vocabulary does this compiler speak? llvm + // payload which archive provides it? emsdk + // + // `to_xim_package` already answers the second from the target; this is the + // field that lets it be SAID. Empty means the family's own payload, which + // is every row but these two, so nothing else's output moves. + std::string payloadName; + std::string spec_str() const { - return std::format("{}@{}", family_name(family), version); + return std::format("{}@{}", + payloadName.empty() ? family_name(family) + : std::string_view(payloadName), + version); } // "gcc@16.1.0" or "gcc@16.1.0 → x86_64-windows-gnu" — user-facing. @@ -96,6 +121,27 @@ struct XimToolchainPackage { } std::string display_spec() const { return displaySpec; } + // WHERE THE FRONTEND LIVES, RELATIVE TO THE PAYLOAD ROOT. + // + // `bin` for every payload that grew up here, and that was hardcoded at the + // one place which composed a root with a bin directory -- with MSVC as a + // named exception four levels deeper. A third and fourth shape make the + // exception list the wrong structure: emsdk keeps `em++` in + // `emscripten/`, and the NDK keeps `clang++` in + // `toolchains/llvm/prebuilt//bin/`. Neither is unusual; what was + // unusual was asking the FAMILY where a PAYLOAD keeps its compiler. + // + // MSVC stays a branch rather than a subdirectory because its path carries + // the toolset version and the host/target arch pair, which is a lookup and + // not a constant. + std::string frontendSubdir = "bin"; + // THE FAMILY, CARRIED RATHER THAN PASSED ALONGSIDE. `payload_frontend` + // took it as a second argument, which every caller had to source from a + // differently-named local; five of them composed `payload->binDir` + // themselves instead and so could not see `frontendSubdir` at all. The + // package is built from a spec that has a family, so carrying it is free + // and removes an argument that could disagree with `pkg`. + Family family = Family::Gcc; }; std::expected @@ -143,8 +189,17 @@ std::filesystem::path toolchain_frontend(const std::filesystem::path& binDir, // here, which is the caller's cue to skip; a wrong LAYOUT and a missing // PAYLOAD had been reporting the same way. std::filesystem::path payload_frontend(const std::filesystem::path& payloadRoot, - const XimToolchainPackage& pkg, - Family family); + const XimToolchainPackage& pkg); + +// The DIRECTORY `payload_frontend` searched, for a message that has to name it. +// +// The five "has no known C++ frontend in " refusals printed +// `payload->binDir`, which was the directory they had composed themselves. Once +// the package decides where its frontend lives, a message naming `bin` would +// be naming a directory nothing looked in -- and that is the failure this +// codebase records most often: the lookup is fixed and the message is not. +std::filesystem::path payload_frontend_dir(const std::filesystem::path& payloadRoot, + const XimToolchainPackage& pkg); // Reverse mapping: an installed `xim-x-` payload directory back to its // (family, target) identity. nullopt for non-toolchain xpkgs (ninja, glibc, @@ -227,6 +282,11 @@ bool needs_linux_sysroot_payloads(const triple::Triple& target); // that the availability side declared impossible). bool host_can_serve(const triple::Triple& target); +// The NDK's own declared minimum API level, read from the installed payload's +// `meta/platforms.json`. 0 when it cannot be read. Definition and the reason +// the number is not a constant are below. +int ndk_min_api_level(const std::filesystem::path& compilerPath); + // xim index names to query for the Available section, with the family each // one contributes versions to. Host-conditional: a host only lists payloads // it can install. @@ -309,16 +369,18 @@ parse_toolchain_spec(std::string compilerArg, auto norm = compat::normalize_spec(compilerArg, versionArg); if (!norm) { return std::unexpected(std::format( - "unknown toolchain '{}' (expected gcc | llvm | msvc, or a " - "supported alias like mingw / musl-gcc)", compilerArg)); + "unknown toolchain '{}' (expected gcc | llvm | msvc | emsdk | " + "android-ndk, or a supported alias like mingw / musl-gcc)", + compilerArg)); } ToolchainSpec spec; if (norm->family == "llvm") spec.family = Family::Llvm; else if (norm->family == "msvc") spec.family = Family::Msvc; else spec.family = Family::Gcc; - spec.version = std::move(norm->version); - spec.target = std::move(norm->target); + spec.version = std::move(norm->version); + spec.target = std::move(norm->target); + spec.payloadName = std::move(norm->payload); // `@system` IS NOT A GENERAL SPELLING, and refusing it here is the point. // @@ -390,10 +452,69 @@ bool gcc_native_payload_is_musl(std::string_view hostArch, bool isLinux, || (target.os == "linux" && target.arch == hostArch); } +// The NDK's own name for the HOST it runs on, which is the directory component +// under `toolchains/llvm/prebuilt/`. Upstream ships `linux-x86_64`, +// `darwin-x86_64` (a universal binary, so Apple silicon reads it too) and +// `windows-x86_64`. Not the target -- a Linux x86_64 machine building for +// aarch64 still reads `linux-x86_64`. +std::string ndk_host_tag() { + if constexpr (mcpp::platform::is_windows) return "windows-x86_64"; + else if constexpr (mcpp::platform::is_macos) return "darwin-x86_64"; + else return "linux-x86_64"; +} + +// THE NDK'S OWN MINIMUM API LEVEL, READ FROM THE PAYLOAD. +// +// Android's API level is NOT OPTIONAL and mcpp cannot leave it out. bionic's +// own stops the build: +// +// sys/cdefs.h:365:2: error: Unversioned target triples are not supported! +// +// So a project that declares no `min_api_level` still needs a level, and the +// question is where the number comes from. Not from a constant compiled in +// here: this repository has recorded more than once that a version written +// into a comment becomes a version written into a diagnostic and then into +// somebody's install command, and the NDK's floor moves with the NDK. The +// payload answers for itself -- `meta/platforms.json` is upstream's own +// declaration of the range it supports, `{"min": 21, "max": 37}` for r30 -- +// and reading it means a newer NDK changes the default by being installed +// rather than by being edited into this file. +// +// Returns 0 when the file is absent or unreadable, which the caller turns into +// a refusal naming `min_api_level`. A guessed level would be worse than the +// refusal: it selects which bionic symbols exist, so guessing produces an +// artefact that links here and fails to load on a device. +int ndk_min_api_level(const std::filesystem::path& compilerPath) { + // `/toolchains/llvm/prebuilt//bin/clang++` -- walk up rather + // than counting components, because the count is exactly the kind of fact + // that changes silently when a layout does. + std::error_code ec; + for (auto dir = compilerPath.parent_path(); + !dir.empty() && dir != dir.parent_path(); + dir = dir.parent_path()) { + auto meta = dir / "meta" / "platforms.json"; + if (!std::filesystem::exists(meta, ec)) continue; + std::ifstream in(meta); + if (!in) return 0; + try { + auto j = nlohmann::json::parse(in, nullptr, false); + if (j.is_discarded() || !j.contains("min")) return 0; + auto min = j["min"]; + if (!min.is_number_integer()) return 0; + auto level = min.get(); + return level> 0 ? level : 0; + } catch (...) { + return 0; + } + } + return 0; +} + XimToolchainPackage to_xim_package(const ToolchainSpec& spec) { XimToolchainPackage pkg; pkg.displaySpec = spec.display(); pkg.ximVersion = spec.version; + pkg.family = spec.family; if (spec.family == Family::Msvc) { // `xim:msvc@`. Only reached for a VERSIONED spec — @@ -413,10 +534,53 @@ XimToolchainPackage to_xim_package(const ToolchainSpec& spec) { return pkg; } if (spec.family == Family::Llvm) { - // ONE PAYLOAD. The `openkal-llvm` spelling normalises to this family and - // installs nothing of its own — it is a statement about where the - // TARGET SIDE comes from, and the compiler is the llvm payload either - // way. A user who has one has both. + // THE TARGET DECIDES THE PAYLOAD HERE TOO, and it did not used to. + // + // This returned the generic llvm payload unconditionally, which is + // right for every target llvm itself serves and wrong for the two that + // arrive with their own clang. `em++` and the NDK's `clang++` ARE + // clang -- same family, same flag vocabulary, same `import std` path + // -- and each is a clang whose target is fixed by its payload, which + // is the shape `src/toolchain/hostflags.cppm` already describes: + // "every hosted cross this build tool could do was served by a payload + // whose driver had exactly one target". So the family stays `Llvm` and + // no fourth value is invented; what changes is which package answers. + const auto& lt = spec.target; + + if (lt.os == "emscripten") { + // `em++` is a `#!/bin/sh` wrapper beside the Python it execs, in + // `emscripten/` rather than `bin/` -- `bin/` holds the raw clang, + // which would compile for wasm and then link like an ordinary + // clang, producing a `.wasm` with no JavaScript and none of + // Emscripten's own glue. Naming the wrapper is the whole point. + pkg.ximName = "emsdk"; + pkg.frontendSubdir = "emscripten"; + pkg.frontendCandidates = { "em++", "emcc" }; + return pkg; + } + + if (lt.is_android()) { + // One NDK payload serves every Android arch and API level: the + // arch arrives as `--target=-linux-android` on the + // command line, not as a different package. The host tuple in the + // path is the HOST's, not the target's -- a Linux x86_64 machine + // cross-compiling for aarch64 still reads + // `prebuilt/linux-x86_64/`. + pkg.ximName = "android-ndk"; + pkg.frontendSubdir = std::format("toolchains/llvm/prebuilt/{}/bin", + ndk_host_tag()); + if constexpr (mcpp::platform::is_windows) { + pkg.frontendCandidates = { "clang++.exe", "clang++" }; + } else { + pkg.frontendCandidates = { "clang++", "clang" }; + } + return pkg; + } + + // ONE PAYLOAD for everything else. The `openkal-llvm` spelling + // normalises to this family and installs nothing of its own — it is a + // statement about where the TARGET SIDE comes from, and the compiler is + // the llvm payload either way. A user who has one has both. pkg.ximName = mcpp::toolchain::llvm::package_name(); pkg.frontendCandidates = mcpp::toolchain::llvm::frontend_candidates(); return pkg; @@ -530,10 +694,22 @@ std::filesystem::path toolchain_frontend(const std::filesystem::path& binDir, return {}; } +std::filesystem::path payload_frontend_dir(const std::filesystem::path& payloadRoot, + const XimToolchainPackage& pkg) { + if (pkg.family == Family::Msvc) { + // The lookup's own answer, so the message names the toolset directory + // rather than a path this code would have guessed. + if (auto inst = mcpp::toolchain::msvc::installation_at(payloadRoot, + pkg.ximVersion)) + return inst->clPath.parent_path(); + return payloadRoot / "VC" / "Tools" / "MSVC" / pkg.ximVersion; + } + return payloadRoot / pkg.frontendSubdir; +} + std::filesystem::path payload_frontend(const std::filesystem::path& payloadRoot, - const XimToolchainPackage& pkg, - Family family) { - if (family == Family::Msvc) { + const XimToolchainPackage& pkg) { + if (pkg.family == Family::Msvc) { // Same resolution the install and build paths use, so the three // cannot disagree about where an msvc payload keeps its compiler. // below, which is the llvm payload's shape.) @@ -542,7 +718,7 @@ std::filesystem::path payload_frontend(const std::filesystem::path& payloadRoot, return inst->clPath; return {}; } - return toolchain_frontend(payloadRoot / "bin", pkg); + return toolchain_frontend(payloadRoot / pkg.frontendSubdir, pkg); } std::optional identify_xim_payload(std::string_view ximDirName) { @@ -602,6 +778,54 @@ bool needs_linux_sysroot_payloads(const triple::Triple& target) { bool host_can_serve(const triple::Triple& target) { if (target.empty()) return true; // host target + // AN SDK THAT SHIPS ITS OWN SYSROOT IS SERVED WHERE THE SDK IS PUBLISHED, + // AND THE ARCH IN THE TRIPLE IS THE GUEST'S. + // + // Every branch below reasons about a cross payload per host arch, because + // that is how a compiler targeting another Linux or another Windows is + // published here. An Emscripten or Android SDK is published per HOST and + // serves every guest arch from one archive: one `xim:emsdk` compiles for + // wasm32 regardless of the machine's arch, and one NDK serves both Android + // arches from a single `--target=-linux-android`. + // + // Without this the wasm row's own pin was not enough. Measured: with + // `emsdk@6.0.9` in the table, `mcpp build --target wasm32-emscripten` + // still answered "No toolchain payload here produces it" and listed + // seventeen servable targets -- because `target.os` is neither "linux" nor + // a Windows form, so control reached a `return false` written for triples + // nobody publishes a payload for. + // + // "WHEREVER" WAS TOO BROAD ONCE, AND THE TARGET MATRIX IS WHAT CAUGHT IT. + // The first version returned true unconditionally, which is the same + // mistake as the branches it sits above: a predicate correct about the + // objects its author had in mind. It was then narrowed to + // `mcpp::platform::is_linux`, because `xim:emsdk` and `xim:android-ndk` + // both declared ONLY an `xpm.linux` table -- so on macOS or Windows there + // was no payload to install and the honest answer was the same + // `host-cannot-serve` every other unpublished combination gets. That + // comment named its own expiry: "when a darwin or windows NDK lands in the + // index -- upstream publishes both -- this is the one line that changes." + // + // IT HAS LANDED, SO THIS IS THAT LINE. Both packages now declare + // `xpm.linux`, `xpm.macosx` and `xpm.windows`, and the index's own + // per-host install jobs are the measurement rather than the declaration: + // on macOS and Windows each payload downloads, extracts, passes its + // recipe's compiler probe and registers its shims. Two host assumptions + // inside those recipes were found by exactly those jobs and fixed there, + // which is where a host-shaped packaging defect belongs -- not here. + // + // Keyed on the target and no longer on the host, because these payloads + // are published per host OS and carry every guest arch: one `xim:emsdk` + // compiles for wasm32 regardless of the machine's arch, and one NDK serves + // both Android arches. The remaining per-host question is whether the + // payload EXISTS, and that is the index's answer to give, not a constant + // compiled into the engine. A row whose pin the index cannot satisfy on + // this host fails at install with the package's own diagnostic, which + // names the payload -- strictly better than this function silently + // deleting the row from `toolchain list`, which reported a target mcpp + // knows as one it has never heard of. + if (target.has_own_sysroot()) return true; + if (target.os == "linux") { if constexpr (mcpp::platform::is_linux) { // "SELF-CONTAINED" IS ABOUT THE PAYLOAD'S CONTENTS, NOT ABOUT diff --git a/tests/e2e/233_bench_matrix.sh b/tests/e2e/233_bench_matrix.sh index beec75ae8..ec48a4f6c 100755 --- a/tests/e2e/233_bench_matrix.sh +++ b/tests/e2e/233_bench_matrix.sh @@ -338,12 +338,33 @@ if fail: print(f"matrix: {len(m['cells'])} cells, {len(m.get('excluded', []))} documented exclusions, " f"baseline={base}, tool pins {m['tools']['cmake']}/{m['tools']['xmake']}/{m['tools']['bazel']}") if uninit: - # Loud, and named. A silent skip here would mean the check that catches a - # stale `hub` never actually runs anywhere, which is how it got missed in - # the first place. The bench workflow checks submodules out and runs this - # test, so the assertion does execute on every change to the suite. - print(f" NOTE: hub/body existence NOT checked for {', '.join(sorted(uninit))} " - f"— submodule(s) not checked out here (`git submodule update --init`)") + # A NOTE CANNOT TURN A JOB RED, and this branch printed one. + # + # The sentence that used to stand here -- "The bench workflow checks + # submodules out and runs this test, so the assertion does execute on every + # change to the suite" -- was false: `.github/workflows/` has no bench + # workflow, and no job checked the submodules out. So the hub/body check + # ran in ZERO CI jobs from the day it was written, and the stale path it + # exists to catch was found by hand on a developer machine (#599). + # + # The two audiences are different and the answers are opposite. A developer + # without submodules must not be blocked by a check about a benchmark they + # are not running. A RUNNER without them is a mis-configured job -- the + # e2e workflow asks for `submodules: recursive` -- and reporting that as a + # note would restore exactly the silence above. + msg = (f"hub/body existence NOT checked for {', '.join(sorted(uninit))} " + f"— submodule(s) not checked out") + if os.environ.get("CI"): + print("FAIL: bench/matrix.json") + print(f" {msg} -- so the check that catches a stale `hub` did not run.") + print(" A job that runs the whole e2e suite needs the pinned trees: add") + print(" - uses: actions/checkout@v4") + print(" with:") + print(" submodules: recursive") + print(" to this job's checkout. Under 10 MB across the three pins, and") + print(" nothing here builds them.") + raise SystemExit(1) + print(f" NOTE: {msg} (`git submodule update --init`)") PY # ── 2: the axis values are ones the harness accepts ──────────────────────── diff --git a/tests/e2e/39_xlings_index_migration.sh b/tests/e2e/39_xlings_index_migration.sh index 41962ad0b..5d4b5b3af 100755 --- a/tests/e2e/39_xlings_index_migration.sh +++ b/tests/e2e/39_xlings_index_migration.sh @@ -30,7 +30,6 @@ search_ttl_seconds = 3600 [build] default_jobs = 0 -default_backend = "ninja" TOML mkdir -p "$MCPP_HOME/registry" diff --git a/tests/e2e/52_local_path_namespaced_index.sh b/tests/e2e/52_local_path_namespaced_index.sh index 7a8eec373..37794fbcb 100755 --- a/tests/e2e/52_local_path_namespaced_index.sh +++ b/tests/e2e/52_local_path_namespaced_index.sh @@ -195,7 +195,6 @@ search_ttl_seconds = 3600 [build] default_jobs = 0 -default_backend = "ninja" [toolchain] default = "gcc@16.1.0" diff --git a/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh b/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh index 963f07340..d598e2788 100644 --- a/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh +++ b/tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh @@ -176,7 +176,6 @@ search_ttl_seconds = 3600 [build] default_jobs = 0 -default_backend = "ninja" [toolchain] default = "gcc@16.1.0" diff --git a/tests/e2e/60_stale_xpkg_cache_reinstall.sh b/tests/e2e/60_stale_xpkg_cache_reinstall.sh index 3ea989026..4c9fc7fee 100644 --- a/tests/e2e/60_stale_xpkg_cache_reinstall.sh +++ b/tests/e2e/60_stale_xpkg_cache_reinstall.sh @@ -101,7 +101,6 @@ search_ttl_seconds = 3600 [build] default_jobs = 0 -default_backend = "ninja" [toolchain] default = "gcc@16.1.0" diff --git a/tests/e2e/638_pack_format_dispatch.sh b/tests/e2e/638_pack_format_dispatch.sh new file mode 100755 index 000000000..1102370ce --- /dev/null +++ b/tests/e2e/638_pack_format_dispatch.sh @@ -0,0 +1,357 @@ +#!/usr/bin/env bash +# requires: gcc +# 638_pack_format_dispatch.sh — `mcpp pack --format ` dispatches to a +# package, and no distribution format lives in the engine. +# +# `tar` and `dir` answer the same question `msi` and `appimage` answer -- what +# shape does the output take -- so they are values of ONE flag, and everything +# past the two the engine owns comes from the resolved graph. What the engine +# adds is the mechanism: a staged tree an artifact action can consume, the rest +# of `[package]` in the build program, and the dispatch itself. +# +# The four properties this holds, each with the wrong answer it excludes: +# +# 1. DECLARE UNCONDITIONALLY, SUBMIT CONDITIONALLY. A build that asks for no +# format still declares one, so `--format bogus` can name what IS +# available. A member that declared only when asked works for its author +# and makes the set unknowable for everyone else. +# 2. THE STAGED TREE IS REAL WHEN THE ACTION RUNS. Asserted from INSIDE the +# action, by listing the tree into the output -- not by the action's exit +# code, because a command that writes nothing and exits 0 is the measured +# failure this whole mechanism exists to prevent. +# 3. A PLAIN BUILD HAS NO DISTRIBUTION EDGE, before or after a pack. +# 4. THE PLACEHOLDER REFUSES OUTSIDE A PACKAGING PASS, rather than expanding +# to an empty string that the tool would accept. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# ── The provider: one build program, both halves of the contract ─────────── +mkdir -p app/src +cd app + +cat> mcpp.toml <<'eof' +[package] +name = "app" +version = "2.5.0" +description = "a program that ships" +license = "Apache-2.0" +authors = ["Ada "] +repo = "https://example.org/app" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +cat> src/main.cpp <<'eof' +#include +int main() { std::puts("app"); return 0; } +EOF + +# The dist step. A separate script because an action's command is an argv with +# no shell assumed, which is the same reason 188_build_actions.sh writes one. +# +# It writes the package VERSION and the staged tree's top level into its +# output, so the assertions below read what the action actually saw rather than +# whether it exited 0. +cat> dist.sh <<'eof' +#!/usr/bin/env bash +set -e +version="1ドル"; stage="2ドル"; out="3ドル" +{ + echo "version=$version" + echo "staged:" + ls -1 "$stage" | sort +}> "$out" +EOF +chmod +x dist.sh + +cat> build.mcpp <<'eof' +import mcpp; +#include +#include +#include +int main() { + // Half one: unconditional. This is what makes the set knowable on a build + // that asked for nothing. + mcpp::provides_pack_format("zap"); + + // Half two: conditional. A plain build must have no such edge. + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; + a.role = "artifact"; + a.description = "zap"; + a.arg((root + "/dist.sh").c_str()) + // The VERSION comes from `[package]`, not from an option the project + // restates: a second copy drifts with nothing able to detect it. + .arg(mcpp::package_version()) + .arg("${mcpp.stage_dir}") + .arg(out.c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +MCPP="${MCPP:-mcpp}" + +graph_dist() { sed -n '2p' "1ドル" | sed 's/.*;dist=//;s/;.*//'; } +find_graph() { find target -name build.ninja | head -1; } + +# ── 1. a plain build declares, and submits nothing ───────────────────────── +"$MCPP" build --release> b1.log 2>&1 || { cat b1.log; echo "FAIL: plain build failed"; exit 1; } +G=$(find_graph) +[ -n "$G" ] || { echo "FAIL: no build.ninja"; exit 1; } +[ "$(graph_dist "$G")" = "none" ] \ + || { sed -n '2p' "$G"; echo "FAIL: a plain build's graph is not dist=none"; exit 1; } +[ -z "$(find target -name 'app.zap' 2>/dev/null)" ] \ + || { echo "FAIL: a plain build produced a distributable"; exit 1; } + +# ── 2. an unknown format is refused, BEFORE the build, naming what exists ── +set +e +"$MCPP" pack --format bogus> b2.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b2.log; echo "FAIL: --format bogus was accepted"; exit 1; } +grep -q "unknown --format 'bogus'" b2.log \ + || { cat b2.log; echo "FAIL: the refusal does not name the value"; exit 1; } +# THE LIST IS THE POINT. A refusal that named a fixed list would be the +# coupling this mechanism removes; this one names the graph's own answer. +grep -q "available in this build: tar, dir, zap" b2.log \ + || { cat b2.log; echo "FAIL: the refusal does not name what is available"; exit 1; } +# Nothing was compiled to find that out. +grep -q "Compiling app" b2.log \ + && { cat b2.log; echo "FAIL: the refusal arrived after a compile"; exit 1; } +# AND NOTHING REACHED STDOUT. This is the path a client hits when it probes an +# mcpp for a capability, so the machine-output contract +# (202_machine_output_contract.sh) requires an empty stdout, a non-empty stderr +# and exit 2. It is asserted here as well because the tension is local to this +# feature: the valid set is a property of the resolved graph, so the refusal +# cannot be decided until prepare has run -- and prepare narrates what it +# resolves. Deciding it later is what broke the contract once. +set +e +so=$("$MCPP" pack --format bogus 2>/dev/null); rc=$? +se=$("$MCPP" pack --format bogus 2>&1>/dev/null) +set -e +[ -z "$so" ] || { echo "FAIL: the refusal wrote to stdout: $(echo "$so" | head -1)"; exit 1; } +[ -n "$se" ] || { echo "FAIL: the refusal said nothing on stderr"; exit 1; } +[ "$rc" = 2 ] || { echo "FAIL: the refusal exited $rc, expected 2"; exit 1; } + +# ── 3. the dispatched format produces a file, and it saw the staged tree ─── +"$MCPP" pack --format zap> b3.log 2>&1 || { cat b3.log; echo "FAIL: pack --format zap failed"; exit 1; } +Z=$(find target -name 'app.zap' | head -1) +[ -n "$Z" ] || { cat b3.log; echo "FAIL: --format zap produced nothing"; exit 1; } +# The engine handed the build program the rest of `[package]`. +grep -qx "version=2.5.0" "$Z" \ + || { cat "$Z"; echo "FAIL: the action was not told the package version"; exit 1; } +# The action ran with a staged tree that already held the program. This is the +# assertion the ordering exists for: an artifact edge is scheduled by ninja and +# the tree is staged by mcpp after the link, so a single-pass design would run +# this against a directory that does not exist. +grep -qx "bin" "$Z" \ + || { cat "$Z"; echo "FAIL: the staged tree had no bin/ when the action ran"; exit 1; } +grep -q "Packed" b3.log \ + || { cat b3.log; echo "FAIL: the produced file was not reported"; exit 1; } +# The staged tree is described by a SIBLING of itself, never a member: a file +# inside it would ship inside every format that packages the directory. +[ -n "$(find target/dist -maxdepth 1 -name '*.stage-manifest' 2>/dev/null)" ] \ + || { echo "FAIL: no stage manifest beside the staged tree"; exit 1; } +[ -z "$(find target/dist -mindepth 2 -name '*.stage-manifest' 2>/dev/null)" ] \ + || { echo "FAIL: the stage manifest landed inside the staged tree"; exit 1; } + +# ── 4. a plain build AFTER the pack still has no distribution edge ───────── +rm -f "$Z" +"$MCPP" build --release> b4.log 2>&1 || { cat b4.log; echo "FAIL: build after pack failed"; exit 1; } +[ "$(graph_dist "$(find_graph)")" = "none" ] \ + || { echo "FAIL: the graph still says dist= after a plain build"; exit 1; } +[ ! -f "$Z" ] \ + || { echo "FAIL: a plain build regenerated the distributable"; exit 1; } + +# ── 5. the placeholder outside a packaging pass is refused ───────────────── +# The same action, ungated. An empty expansion would give the tool the build +# directory root, which exists -- so the mistake would produce a plausible +# artifact instead of a diagnostic. +cd "$TMP" +cp -r app ungated +cd ungated +cat> build.mcpp <<'eof' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}").arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()).submit(); + return 0; +} +EOF +set +e +"$MCPP" build --release> b5.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b5.log; echo "FAIL: an ungated stage_dir built"; exit 1; } +grep -q "this build is not packaging" b5.log \ + || { cat b5.log; echo "FAIL: the refusal does not say why"; exit 1; } +grep -q "provides_pack_format" b5.log \ + || { cat b5.log; echo "FAIL: the refusal does not say what to do instead"; exit 1; } + +# ── 6. a role other than artifact cannot reach the staged tree ───────────── +# GATED, so this is a role refusal and not the one above. An ungated action is +# already refused in the FIRST pass of `mcpp pack`, before there is a staged +# tree, which is why the two cases need different fixtures rather than a +# one-word edit: only a gated action gets far enough for its role to matter. +cd "$TMP" +cp -r app wrongrole +cd wrongrole +cat> build.mcpp <<'eof' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "source"; + a.arg((root + "/dist.sh").c_str()).arg("x").arg("${mcpp.stage_dir}").arg(out.c_str()) + .output(out.c_str()).submit(); + return 0; +} +EOF +set +e +"$MCPP" pack --format zap> b6.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b6.log; echo "FAIL: a source action reached the staged tree"; exit 1; } +grep -q 'other than "artifact"' b6.log \ + || { cat b6.log; echo "FAIL: the role refusal does not name the role"; exit 1; } + +# ── 7. declared and never submitted is a refusal, not a success ──────────── +# THE HALF A MEMBER AUTHOR IS MOST LIKELY TO GET WRONG. A gate that never +# opens leaves a pass that succeeds and produces no package, which reads as +# "packaging is not implemented yet" rather than as a defect in the member. +cd "$TMP" +cp -r app silent +cd silent +cat> build.mcpp <<'eof' +import mcpp; +int main() { mcpp::provides_pack_format("zap"); return 0; } +EOF +set +e +"$MCPP" pack --format zap> b7.log 2>&1 +rc=$? +set -e +[ "$rc" -ne 0 ] || { cat b7.log; echo "FAIL: a format nothing claimed reported success"; exit 1; } +grep -q "no action claimed --format 'zap'" b7.log \ + || { cat b7.log; echo "FAIL: the refusal does not name the unclaimed format"; exit 1; } + +# ── 8. an artifact action that predates the request is not the package ───── +# THE CRITERION IS "WHAT THE REQUEST INTRODUCED", and this is what distinguishes +# it from "any artifact action". A codesign stamp or a size budget is also an +# artifact action and is present whether or not a format was asked for; naming +# one as the distributable would be a wrong answer that looks like a right one. +# +# It is also what an earlier revision got wrong from the other side: the check +# collected only actions naming ${mcpp.stage_dir}, which refused a member that +# packages ONE NAMED PROGRAM and never reads the tree -- the shape the design +# record recommends, after a bind path that resolved to nothing produced a +# valid, empty, 52 KB installer. So this fixture submits both shapes: an +# ungated stamp that must be ignored, and a gated action that names no staged +# tree at all and must still be reported. +cd "$TMP" +cp -r app twoshapes +cd twoshapes +cat> build.mcpp <<'eof' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + const std::string root = mcpp::manifest_dir(); + + // Ungated: present in both passes, so it is not the distributable. + const std::string stamp = std::string(mcpp::out_dir()) + "/size.stamp"; + mcpp::action s; + s.id = "size-budget"; s.role = "artifact"; + s.arg((root + "/dist.sh").c_str()).arg("stamp").arg(root.c_str()).arg(stamp.c_str()) + .input("${mcpp.target_file:app}").output(stamp.c_str()); + s.submit(); + + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + // Gated, and it names NO staged tree: the program arrives through + // ${mcpp.target_file:...} exactly as an MSI's one File row does. + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("named").arg(root.c_str()).arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()); + a.submit(); + return 0; +} +EOF +"$MCPP" pack --format zap> b8.log 2>&1 || { cat b8.log; echo "FAIL: a member that reads no staged tree was refused"; exit 1; } +grep -q "app.zap" b8.log \ + || { cat b8.log; echo "FAIL: the gated action was not reported as the package"; exit 1; } +grep -q "size.stamp" b8.log \ + && { cat b8.log; echo "FAIL: an action predating the request was reported as the package"; exit 1; } +# Both files exist -- the stamp was built, it was simply not the answer. +[ -n "$(find target -name 'size.stamp' 2>/dev/null)" ] \ + || { echo "FAIL: the ungated artifact action did not run at all"; exit 1; } + +# ── 9. a format that names no staged tree survives a staging refusal ────── +# THE CASE macOS FOUND, HELD ON EVERY PLATFORM. +# +# `mcpp pack` refuses to bundle a Mach-O PROGRAM: the built-in closure walk is +# `LD_TRACE_LOADED_OBJECTS`, glibc's mechanism, and dyld ignores it and runs the +# program instead. That refusal is right about the built-in archive and says +# nothing about whether a `.app` bundler can work -- one that names a single +# program needs no closure walk at all. Before this, the refusal happened before +# the dispatch, so EVERY dispatched format was unreachable on that target. +# +# Staging is now a service to the provider rather than a precondition. This leg +# cannot reproduce the Mach-O refusal on Linux, so it holds the property the fix +# rests on instead: a provider that reads no staged tree is reported, and the +# reason travels far enough to reach the placeholder's refusal. +cd "$TMP" +cp -r app nostage +cd nostage +cat> build.mcpp <<'eof' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + // Reads NOTHING from the staged tree: the program arrives through + // ${mcpp.target_file:...}, which is what an `.msi` of one program does. + const std::string root = mcpp::manifest_dir(); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; a.role = "artifact"; + a.arg((root + "/dist.sh").c_str()).arg("named").arg(root.c_str()).arg(out.c_str()) + .input("${mcpp.target_file:app}").output(out.c_str()); + a.submit(); + return 0; +} +EOF +"$MCPP" pack --format zap> b9.log 2>&1 || { cat b9.log; echo "FAIL: a provider that reads no staged tree was refused"; exit 1; } +grep -q "app.zap" b9.log \ + || { cat b9.log; echo "FAIL: the provider was not reported as the package"; exit 1; } +# It really did not consume the tree: no stage manifest edge was added, so the +# action's inputs are the program alone. +grep -q "stage-manifest" b9.log \ + && { cat b9.log; echo "FAIL: a provider that names no tree gained a stage dependency"; exit 1; } +echo "ok: a provider that reads no staged tree is dispatched and reported" + +echo "PASS: 638_pack_format_dispatch" diff --git a/tests/e2e/639_the_scanner_does_not_read_inside_a_comment.sh b/tests/e2e/639_the_scanner_does_not_read_inside_a_comment.sh new file mode 100755 index 000000000..1ef97d604 --- /dev/null +++ b/tests/e2e/639_the_scanner_does_not_read_inside_a_comment.sh @@ -0,0 +1,132 @@ +#!/usr/bin/env bash +# requires: gcc +# 639_the_scanner_does_not_read_inside_a_comment.sh — comment bodies, comment +# tails and raw-string bodies are not code, and the scanner decides which of +# the three a line is in by whichever opener came first. +# +# THE DEFECT WAS BIDIRECTIONAL, and the reported direction was the cheaper one. +# +# The line loop had two passes -- raw strings, then an unconditional +# `find("//")` -- and no block-comment state at any point in the file's history. +# So a block comment whose opener sits on its own line puts the keyword at the +# start of the next line, and the matcher fires inside a comment: +# +# /* +# module (exe) -> error: '(exe)' is not a module name +# */ +# +# With a WELL-FORMED name that refusal does not fire and the result is worse. +# `export module y;` inside a block comment made a plain `.cpp` the recorded +# producer of `gcm.cache/y.gcm`, a BMI the compiler never writes -- and a file +# that legitimately imports `y` was then told `imports must be built before +# being imported`, an ordering problem that does not exist, while the real +# provider was never searched for. +# +# And in the OTHER direction, a commented-out raw-string opener put the +# raw-string pass into a state it could not leave, blanking every following +# line until a `)"` that never comes: +# +# // R"( +# import x; -> invisible to the scanner, visible to gcc +# +# That one is a MISSING DEPENDENCY EDGE: the compile is not ordered after the +# BMI it needs, so it fails under parallelism and passes on a retry. The +# reported form is at least deterministic. +# +# CASES 4 AND 5 ARE WHY THIS IS A FIX AND NOT A MUTE. `/* */ import x;` must +# still record the import, which "skip any line starting with /*" would fail; +# and `"a /* b"` must not open a comment, which "treat every /* as an opener" +# would fail. Both were measured before the fix: the first was already broken +# (a fourth wrong answer the issue did not report), the second was correct only +# because ordinary strings were left alone by a line-start matcher. +set -e + +t=$(mktemp -d); trap 'rm -rf "$t"' EXIT + +# `imported but not provided` is emitted by mcpp's own scanner and by nothing +# else, so it separates "the scanner saw the import" from "the compiler did". +# The module never exists, so the build always fails; what is asserted is the +# WARNING, not the exit code. +scanner_sees_import() { # 1ドル = source text + local d="$t/$RANDOM$RANDOM" + mkdir -p "$d/src" + printf '[package]\nname = "s"\nversion = "0.1.0"\n'> "$d/mcpp.toml" + printf '%b' "1ドル"> "$d/src/main.cpp" + ( cd "$d" && "$MCPP" build 2>&1 ) | grep -q "imported but not provided" +} + +fail=0 +expect_seen() { + if scanner_sees_import "2ドル"; then echo " ok: 1ドル" + else echo "FAIL: 1ドル — the scanner did not record the import"; fail=1; fi +} +expect_unseen() { + if scanner_sees_import "2ドル"; then + echo "FAIL: 1ドル — the scanner recorded an import that is not code"; fail=1 + else echo " ok: 1ドル"; fi +} + +echo "== 639: what is code, and what only looks like it ==" + +# 1. The reported case. A malformed module name inside a comment is not a +# scanner error, and the criterion here is the ERROR -- not the warning the +# other cases use, because there is no import in this file to be seen or +# missed. The fixture is the four lines from the issue, verbatim. +d="$t/refusal"; mkdir -p "$d/src" +printf '[package]\nname = "r"\nversion = "0.1.0"\n'> "$d/mcpp.toml" +printf '/*\n module (exe)\n*/\nint main() { return 0; }\n'> "$d/src/main.cpp" +if out=$( cd "$d" && "$MCPP" build 2>&1 ) && ! grep -q "scanner errors" <<<"$out"; then + echo " ok: a block comment's body is not a module declaration" +else + echo "FAIL: the four-line file from the issue was refused:" + grep -m2 -A1 "scanner errors" <<<"$out" | sed 's/^/ /' + fail=1 +fi + +# 2. The same shape with a name that PARSES, so the refusal above cannot fire +# and an import inside a comment is recorded instead. +expect_unseen "a block comment's body is not an import" \ + '/*\n import x;\n*/\nint main() { return 0; }\n' + +# 3. The other direction: a commented-out raw-string opener must not swallow +# the code after it. +expect_seen "a // comment does not open a raw string" \ + '// R"(\nimport x;\nint main() { return 0; }\n' +expect_seen "a block comment does not open a raw string" \ + '/*\n R"(\n*/\nimport x;\nint main() { return 0; }\n' + +# 4. A closed block comment leaves the rest of its line as code. +expect_seen "code after a closed block comment is still code" \ + '/* */ import x;\nint main() { return 0; }\n' + +# 5. An ordinary string is not a comment opener. +expect_seen "a /* inside a string literal opens nothing" \ + 'const char* s = "a /* b";\nimport x;\nint main() { return 0; }\n' + +# 6. A raw string still hides what it contains -- the property the raw-string +# pass existed for, which this must not have cost. +expect_unseen "a raw-string body is still not code" \ + 'const char* s = R"(\nimport x;\n)";\nint main() { return 0; }\n' + +# 7. THE GRAPH, not the warning. The phantom producer is the expensive form of +# this defect, and its criterion is an edge that must not exist: the +# generated ninja file must name no BMI output for a module that only a +# comment mentions. Asserted on the graph rather than on the build result, +# because the build SUCCEEDED while the graph was wrong. +d="$t/graph"; mkdir -p "$d/src" +printf '[package]\nname = "g"\nversion = "0.1.0"\n'> "$d/mcpp.toml" +printf '/*\n export module y;\n*/\nint main() { return 0; }\n'> "$d/src/main.cpp" +( cd "$d" && "$MCPP" build>/dev/null 2>&1 ) || true +graph=$(find "$d/target" -name build.ninja | head -1) +if [ -z "$graph" ]; then + echo "FAIL: no build.ninja was generated"; fail=1 +elif grep -q "y\.gcm\|y\.pcm\|y\.ifc" "$graph"; then + echo "FAIL: the graph names a BMI for 'y', which only a comment mentions:" + grep -n "y\.gcm\|y\.pcm\|y\.ifc" "$graph" | head -3 | sed 's/^/ /' + fail=1 +else + echo " ok: no BMI edge for a module named only inside a comment" +fi + +if [ "$fail" -ne 0 ]; then echo "FAIL: 639"; exit 1; fi +echo "PASS: 639" diff --git a/tests/e2e/640_a_capability_pin_explains_its_own_row.sh b/tests/e2e/640_a_capability_pin_explains_its_own_row.sh new file mode 100755 index 000000000..fe44bf1e4 --- /dev/null +++ b/tests/e2e/640_a_capability_pin_explains_its_own_row.sh @@ -0,0 +1,132 @@ +#!/usr/bin/env bash +# requires: gcc +# 640_a_capability_pin_explains_its_own_row.sh — a row whose pin is a +# capability refuses a declared toolchain that cannot emit it, and the reason +# names THAT row. +# +# THE RULE IS SHARED AND THE REASON IS NOT. `prepare.cppm` says so in its own +# comment -- "one sentence covering both would be wrong about one of them: a +# PE+musl target is not bare metal, and a reader told it is stops reading" -- +# and then a third row was added without a third reason, so the sentence became +# wrong about the new one. Measured: `--target wasm32-emscripten` with a +# declared gcc was refused correctly and explained with "No gcc payload emits a +# PE with a musl C library", which is true about a different row. +# +# AND THE GATE ASKED THE WRONG QUESTION. It tested `family != Llvm`, which was +# right while every capability-pinned row pinned llvm. `wasm32-emscripten` pins +# `emsdk@6.0.9`, and emsdk normalises to the llvm family because `em++` IS +# clang -- so a declared `llvm@22.1.8` passed the gate, was never refused, and +# resolved the generic llvm payload for a target it cannot emit. Case 4 is that +# one, and it is the case a reader would not think to write. +set -e + +t=$(mktemp -d); trap 'rm -rf "$t"' EXIT +fail=0 + +# Each row's reason must appear for ITS target and for no other. The probe is a +# substring of the sentence rather than the whole of it, so rewording stays +# free while the pairing stays asserted. +check() { # target declared-toolchain expected-phrase label + local target=1ドル tc=2ドル phrase=3ドル label=4ドル + local d="$t/$RANDOM$RANDOM"; mkdir -p "$d/src" + printf '[package]\nname = "c"\nversion = "0.1.0"\n\n[toolchain]\nlinux = "%s"\n' "$tc"> "$d/mcpp.toml" + printf 'int main(){return 0;}\n'> "$d/src/main.cpp" + local out + out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target "$target" 2>&1 ) || true + if ! grep -q "cannot be emitted by" <<<"$out"; then + echo "FAIL: $label — not refused at all" + grep -m3 -E "^(error|warning)" <<<"$out" | sed 's/^/ /' + fail=1; return + fi + if grep -qF "$phrase" <<<"$out"; then + echo " ok: $label" + else + echo "FAIL: $label — the refusal does not carry this row's reason" + echo " wanted: $phrase" + grep -A3 "cannot be emitted by" <<<"$out" | sed 's/^/ /' + fail=1 + fi +} + +echo "== 640: the rule is shared, the reason is the row's ==" + +# 1-3. each row's own sentence, for a declared gcc. +check wasm32-emscripten gcc@16.1.0 "Nothing but Emscripten emits WebAssembly" "wasm names Emscripten" +check riscv64-none-elf gcc@16.1.0 "no per-host cross payload" "bare metal names the cross payload" +check x86_64-windows-musl gcc@16.1.0 "PE with a musl C library" "PE+musl names the C library" + +# 3b. AND IT HAPPENED A SECOND TIME, with Android. The row gained a pin, became +# a capability row, and the reason chain still had three arms -- so the +# refusal explained it with the PE+musl sentence, the identical wrong answer +# case 1 was written for. +check aarch64-linux-android gcc@16.1.0 "An Android target needs bionic" "android names bionic" +check x86_64-linux-android gcc@16.1.0 "An Android target needs bionic" "android names bionic (x86_64)" + +# 4. THE GATE. `llvm@22.1.8` is the llvm family, and so is emsdk -- so a family +# test cannot separate them and this declaration used to pass unrefused. +check wasm32-emscripten llvm@22.1.8 "Nothing but Emscripten emits WebAssembly" "a declared llvm is refused too" +# The NDK normalises to the llvm family for the same reason, so the same hole +# would have existed for Android. A declared `llvm@22.1.8` names a real +# compiler that emits aarch64 ELF perfectly well -- what it cannot supply is +# bionic, which is why this row is a capability at all. +check aarch64-linux-android llvm@22.1.8 "An Android target needs bionic" "a declared llvm is refused for android too" + +# 5. And the sentence names the row's OWN pin rather than a fixed word: the +# closing line used to read "The row names llvm as a capability" on every +# row, which is wrong for the one pinned to emsdk. +d="$t/pin"; mkdir -p "$d/src" +printf '[package]\nname = "c"\nversion = "0.1.0"\n\n[toolchain]\nlinux = "gcc@16.1.0"\n'> "$d/mcpp.toml" +printf 'int main(){return 0;}\n'> "$d/src/main.cpp" +out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target wasm32-emscripten 2>&1 ) || true +if grep -q "names .emsdk@6.0.9. as a capability" <<<"$out"; then + echo " ok: the closing line names this row's pin" +else + echo "FAIL: the closing line does not name emsdk@6.0.9" + grep -A5 "cannot be emitted by" <<<"$out" | sed 's/^/ /' + fail=1 +fi + +# 6. EXHAUSTIVE, BECAUSE ADDING AN ARM IS WHAT KEEPS FAILING. +# +# Cases 1-5 each name a row a reader thought of. The defect both times was a +# row NOBODY thought of falling into a final `else` written as another row's +# answer, and no per-row test can catch that. So: take every pinned row from +# the engine's own vocabulary, declare a toolchain that is not its pin, and +# assert the PE+musl sentence appears for exactly one of them. +# +# The denominator comes from `toolchain list`, so a row added tomorrow is in it +# without this file being edited. +echo "== 640/6: the PE+musl sentence belongs to exactly one row ==" +pinned=$( "$MCPP" toolchain list --format json 2>/dev/null \ + | tr ',' '\n' | grep -o '"target": *"[^"]*"' | sed 's/.*: *"//;s/"//' ) +[ -n "$pinned" ] || { echo "FAIL: toolchain list named no targets"; exit 1; } +peMusl=0; examined=0 +for target in $pinned; do + d="$t/x-$target"; mkdir -p "$d/src" + printf '[package]\nname = "c"\nversion = "0.1.0"\n\n[toolchain]\nlinux = "gcc@16.1.0"\n'> "$d/mcpp.toml" + printf 'int main(){return 0;}\n'> "$d/src/main.cpp" + out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target "$target" 2>&1 ) || true + grep -q "cannot be emitted by" <<<"$out" || continue # not a capability row + examined=$((examined + 1)) + if grep -qF "PE with a musl C library" <<<"$out"; then + peMusl=$((peMusl + 1)) + if [ "$target" != "x86_64-windows-musl" ]; then + echo "FAIL: $target is explained with the PE+musl sentence" + fail=1 + fi + fi +done +echo " examined $examined capability-pinned rows of $(wc -w <<<"$pinned") targets" +if [ "$examined" -lt 4 ]; then + echo "FAIL: only $examined capability rows were reached — the enumeration is too small to be evidence" + fail=1 +fi +if [ "$peMusl" -ne 1 ]; then + echo "FAIL: the PE+musl sentence was printed for $peMusl rows, expected exactly 1" + fail=1 +else + echo " ok: exactly one row is explained by the PE+musl sentence" +fi + +if [ "$fail" -ne 0 ]; then echo "FAIL: 640"; exit 1; fi +echo "PASS: 640" diff --git a/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh b/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh new file mode 100755 index 000000000..9f8801a40 --- /dev/null +++ b/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh @@ -0,0 +1,135 @@ +#!/usr/bin/env bash +# requires: gcc +# 641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh — the vocabulary +# half of the Android and iOS work, which is the half a runner without a 704 MB +# NDK can still assert. Nothing here installs a payload. +# +# WHAT THIS FILE DELIBERATELY DOES NOT ASSERT: that an Android artefact builds. +# That was measured by hand (linux-x86_64, xim:android-ndk 30.0.16248370, a +# source that imports std and no project vocabulary beyond `--target`): +# +# aarch64-linux-android -> ELF 64-bit LSB pie, ARM aarch64, +# interpreter /system/bin/linker64 +# x86_64-linux-android -> ELF 64-bit LSB pie, x86-64, same interpreter +# +# and it is what the rows' `preview` tier records. An e2e that needed the +# payload would skip on every shard, which this repository has already paid for +# once: `# requires: llvm` tests ran on no CI job at all while reporting green. +set -e + +t=$(mktemp -d); trap 'rm -rf "$t"' EXIT +fail=0 + +pkg() { # dir [extra manifest lines...] + local d=1ドル; shift + mkdir -p "$d/src" + printf '[package]\nname = "c"\nversion = "0.1.0"\n'> "$d/mcpp.toml" + for line in "$@"; do printf '%s\n' "$line">> "$d/mcpp.toml"; done + printf 'int main(){return 0;}\n'> "$d/src/main.cpp" +} + +echo "== 641: the rows are wired, and a spelling that exists is not 'unknown' ==" + +# 1. BOTH ANDROID ROWS NAME THEIR PAYLOAD. A row whose tier moved without a pin +# is reachable only through an explicit `[target.X] toolchain` override, +# which is the escape hatch rather than the support claim. +list=$( "$MCPP" toolchain list --format json 2>/dev/null ) +for target in aarch64-linux-android x86_64-linux-android; do + if grep -q "\"target\": *\"$target\"" <<<"$list" \ + && tr ',' '\n' <<<"$list" | grep -A6 "\"target\": *\"$target\"" \ + | grep -q "android-ndk"; then + echo " ok: $target names the NDK payload" + else + echo "FAIL: $target does not name android-ndk in toolchain list" + tr ',' '\n' <<<"$list" | grep -A6 "\"target\": *\"$target\"" | sed 's/^/ /' + fail=1 + fi +done + +# 2. AND THEY ARE NO LONGER `planned`. This is the property the user's question +# was about, and it is separate from (1): a row can carry a pin and still be +# refused by the tier gate. +for target in aarch64-linux-android x86_64-linux-android; do + d="$t/tier-$target"; pkg "$d" + out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target "$target" 2>&1 ) || true + if grep -q "not yet supported (planned)" <<<"$out"; then + echo "FAIL: $target is still refused as planned" + fail=1 + else + echo " ok: $target is not refused for its tier" + fi +done + +# 3. THE SIMULATOR IS A ROW, so its spelling resolves. Before it existed, +# `--target aarch64-ios-sim` answered `unknown target`, which was false: the +# vocabulary has the device row and the simulator is a different target, not +# an unspellable one. +for target in aarch64-ios-sim x86_64-ios-sim; do + d="$t/sim-$target"; pkg "$d" + out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target "$target" 2>&1 ) || true + if grep -q "unknown target" <<<"$out"; then + echo "FAIL: $target reported as unknown; it is a registered row" + fail=1 + elif grep -q "not yet supported (planned)" <<<"$out"; then + echo " ok: $target says planned, naming the row" + else + echo "FAIL: $target refused for neither reason" + grep -m2 -E "^error" <<<"$out" | sed 's/^/ /' + fail=1 + fi +done + +# 4. AN EFFECTIVE TRIPLE mcpp PRINTS ITSELF MUST PARSE BACK. +# `Target aarch64-linux-android -> aarch64-unknown-linux-android21` is a +# line mcpp writes; pasting it back used to answer `unknown target`, because +# the env match read `k == "android"` and the API level rides that segment. +d="$t/effective"; pkg "$d" +out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target aarch64-unknown-linux-android21 2>&1 ) || true +if grep -q "unknown target" <<<"$out"; then + echo "FAIL: the effective triple mcpp prints does not parse back" + grep -m2 -E "^error" <<<"$out" | sed 's/^/ /' + fail=1 +else + echo " ok: aarch64-unknown-linux-android21 parses to the canonical row" +fi + +# 5. A BARE `aarch64-linux` IS NEVER COMPLETED TO ANDROID. The rows sit on the +# same `arch-os` prefix because the kernel IS Linux, and that is the whole +# reason every Linux-shaped answer in the tree is right about them. It does +# not make bionic a candidate C library for a request that named none. +d="$t/bare"; pkg "$d" +out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target aarch64-linux 2>&1 ) || true +if grep -q "android" <<<"$out"; then + echo "FAIL: a bare aarch64-linux request mentioned android" + grep -m4 -E "android" <<<"$out" | sed 's/^/ /' + fail=1 +else + echo " ok: a bare aarch64-linux request never mentions android" +fi + +# 6. `min_api_level` IS A MANIFEST KEY WITH A FLOOR, and it is refused where a +# reader can act on it. Manifest-level, so no payload is needed: the check +# runs before any toolchain is resolved. +d="$t/apilevel"; pkg "$d" "" "[target.aarch64-linux-android]" "min_api_level = 0" +out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target aarch64-linux-android 2>&1 ) || true +if grep -q "min_api_level must be a positive integer" <<<"$out"; then + echo " ok: a non-positive min_api_level is refused naming the key" +else + echo "FAIL: min_api_level = 0 was not refused with its own message" + grep -m3 -E "^error" <<<"$out" | sed 's/^/ /' + fail=1 +fi + +# And a legal one is accepted as far as the manifest is concerned -- the build +# may then fail for want of a payload, which is a different sentence. +d="$t/apilevel-ok"; pkg "$d" "" "[target.aarch64-linux-android]" "min_api_level = 24" +out=$( cd "$d" && MCPP_NO_AUTO_INSTALL=1 "$MCPP" build --target aarch64-linux-android 2>&1 ) || true +if grep -q "min_api_level must be" <<<"$out"; then + echo "FAIL: a legal min_api_level = 24 was refused by the manifest" + fail=1 +else + echo " ok: min_api_level = 24 is accepted by the manifest" +fi + +if [ "$fail" -ne 0 ]; then echo "FAIL: 641"; exit 1; fi +echo "PASS: 641" diff --git a/tests/matrix/expected.tsv b/tests/matrix/expected.tsv index 8fb38b869..66af3ddb2 100644 --- a/tests/matrix/expected.tsv +++ b/tests/matrix/expected.tsv @@ -46,10 +46,14 @@ # 那台机器。 # # 各台格数不同,而这是事实不是遗漏: -# linux-x86_64 40 gcc + llvm ×ばつ 12 目标(payload 24 / graph 16) -# linux-aarch64 16 只有 musl-gcc —— llvm 在非 x86_64 Linux 上被显式延缓 -# macos-arm64 20 只有 llvm ×ばつ 12 目标(payload 12 / graph 8) -# windows-x86_64 40 llvm + msvc@system +# linux-x86_64 74 gcc + llvm(payload 50 / graph 24) +# linux-aarch64 33 只有 musl-gcc —— llvm 在非 x86_64 Linux 上被显式延缓 +# macos-arm64 37 只有 llvm(payload 25 / graph 12) +# windows-x86_64 74 llvm + msvc@system +# +# 这几个数字是**声明**,和表里的行一样参与比对(compare.sh 先比总数再比每一格), +# 所以加了目标行就必须同时改它们。2026-09-11 加入方案 §3 的四个平台行时,四台 +# 宿主各 +12 格:两种体系 ×ばつ 该宿主声明的编译器 ×ばつ 4 个目标。 graph linux-aarch64 aarch64-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 aarch64-linux-musl gcc@16.1.0 - - - - - unsupported layer-requirement graph linux-aarch64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned @@ -220,3 +224,89 @@ payload windows-x86_64 x86_64-windows-msvc llvm@22.1.8 x86_64-pc-windows-msvc no payload windows-x86_64 x86_64-windows-msvc msvc@system x86_64-pc-windows-msvc none msvc(payload) (payload) - ok none payload windows-x86_64 x86_64-windows-musl llvm@22.1.8 - - - - - unsupported host-cannot-serve payload windows-x86_64 x86_64-windows-musl msvc@system - - - - - unsupported capability-pin + +# ── 方案 §3 的三个平台:词表里有,还没有任何东西接线 ────────────────────── +# +# 四行全部 `planned`,于是全部十二格(两种体系 ×ばつ 四台宿主的编译器轴)都是 +# `unsupported / tier-planned`。这些格子不是占位:它们断言的是**拒绝的形状**—— +# 一个 planned 目标必须报「词表里有这一行,还没有接线」,而不是 `unknown target` +# (那是假的),也不是一次解析通过却什么都没建出来的构建(那更糟)。 +# +# 每一行接上线的时候,这里对应的那一格会从 tier-planned 变成别的东西,而这张表 +# 会因此变红 —— 那正是它该做的事。缺的东西每一处都是**载荷**,不是引擎: +# aarch64-linux-android / x86_64-linux-android xim:android-ndk +# aarch64-ios iPhoneOS SDK(先要一次许可判断) +# wasm32-emscripten xim:emsdk,以及 #597 的目标模型 +# +graph linux-aarch64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +graph macos-arm64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +graph windows-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +graph windows-x86_64 aarch64-linux-android msvc@system - - - - - unsupported capability-pin +payload linux-aarch64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 aarch64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +payload macos-arm64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +payload windows-x86_64 aarch64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +payload windows-x86_64 aarch64-linux-android msvc@system - - - - - unsupported capability-pin +graph linux-aarch64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +graph macos-arm64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +graph windows-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +graph windows-x86_64 x86_64-linux-android msvc@system - - - - - unsupported capability-pin +payload linux-aarch64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 x86_64-linux-android gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +payload macos-arm64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +payload windows-x86_64 x86_64-linux-android llvm@22.1.8 - - - - - unsupported capability-pin +payload windows-x86_64 x86_64-linux-android msvc@system - - - - - unsupported capability-pin +graph linux-aarch64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-aarch64 aarch64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-aarch64 x86_64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 x86_64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph linux-x86_64 aarch64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +graph linux-x86_64 x86_64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 aarch64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +graph macos-arm64 x86_64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 x86_64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios msvc@system - - - - - unsupported tier-planned +graph windows-x86_64 aarch64-ios-sim msvc@system - - - - - unsupported tier-planned +graph windows-x86_64 x86_64-ios-sim msvc@system - - - - - unsupported tier-planned +payload linux-aarch64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-aarch64 aarch64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-aarch64 x86_64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 x86_64-ios-sim gcc@16.1.0 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload linux-x86_64 aarch64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +payload linux-x86_64 x86_64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 aarch64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +payload macos-arm64 x86_64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 x86_64-ios-sim llvm@22.1.8 - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios msvc@system - - - - - unsupported tier-planned +payload windows-x86_64 aarch64-ios-sim msvc@system - - - - - unsupported tier-planned +payload windows-x86_64 x86_64-ios-sim msvc@system - - - - - unsupported tier-planned +graph linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported capability-pin +graph macos-arm64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported capability-pin +graph windows-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported capability-pin +graph windows-x86_64 wasm32-emscripten msvc@system - - - - - unsupported capability-pin +payload linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported capability-pin +payload macos-arm64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported capability-pin +payload windows-x86_64 wasm32-emscripten llvm@22.1.8 - - - - - unsupported capability-pin +payload windows-x86_64 wasm32-emscripten msvc@system - - - - - unsupported capability-pin diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index 3943d9488..90f22ea86 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -208,7 +208,8 @@ TEST(BuildDirectives, SerializeDeserializeRoundTrip) { "mcpp:include-dir=inc\n" "mcpp:include-dir-after=after\n" "mcpp:fact=widget.driver=1.2\n" - "mcpp:floor=widget.driver>= 1.0\n"); + "mcpp:floor=widget.driver>= 1.0\n" + "mcpp:pack-format=appimage\n"); std::ostringstream os; dirs::serialize(os, d); @@ -258,6 +259,44 @@ TEST(BuildDirectives, FactsAndFloorsAreClaimsThatFoldIntoRuntimeDeclarations) { EXPECT_EQ(m.runtimeConfig.requirements[0].phase, "build"); } +TEST(BuildDirectives, APackFormatIsANameCarriedAndNotInterpreted) { + auto d = parse("mcpp:pack-format=appimage\n" + "mcpp:pack-format=msi\n"); + EXPECT_EQ(d.at(dirs::Slot::PackFormats), + (std::vector{"appimage", "msi"})); + + mcpp::manifest::Manifest m; + dirs::apply(m, d); + EXPECT_EQ(m.buildConfig.packFormats, + (std::vector{"appimage", "msi"})); + // The engine holds the DISPATCH and no format, so a name reaches neither + // the compile line nor the link line. A row that leaked into either would + // put dpkg's or WiX's vocabulary on a command line. + EXPECT_TRUE(m.buildConfig.cflags.empty()); + EXPECT_TRUE(m.buildConfig.cxxflags.empty()); + EXPECT_TRUE(m.buildConfig.ldflags.empty()); + EXPECT_TRUE(m.buildConfig.sources.empty()); + EXPECT_TRUE(m.buildConfig.actions.empty()); +} + +TEST(BuildDirectives, APackFormatDeclarationIsPersisted) { + // THE HALF THAT WOULD OTHERWISE BE MISSED. A build program's result is + // cached and a hit does not re-run it, and the pass that READS this set is + // `mcpp pack`, which is never a project's first build. A declaration that + // was not persisted would therefore be present exactly once and absent + // every time it mattered, and `--format ` would refuse naming + // nothing. + // + // `SerializeDeserializeRoundTrip` above asserts the round trip for every + // tagged row, so this only has to hold the field that puts the row in that + // set -- which is the field a new row is most likely to be added without. + const auto* def = dirs::find_by_wire("pack-format"); + ASSERT_NE(def, nullptr); + EXPECT_FALSE(def->tag.empty()) << "a tag-less row is not replayed on a cache hit"; + EXPECT_EQ(def->slot, dirs::Slot::PackFormats); + EXPECT_EQ(def->scope, dirs::Scope::Claim); +} + TEST(BuildDirectives, AClaimReachesNeitherCompileNorLink) { auto d = parse("mcpp:fact=widget.driver=1.2\n" "mcpp:floor=widget.driver>= 2.0\n"); diff --git a/tests/unit/test_cache_key.cpp b/tests/unit/test_cache_key.cpp index d0b9c2736..adc500412 100644 --- a/tests/unit/test_cache_key.cpp +++ b/tests/unit/test_cache_key.cpp @@ -132,8 +132,21 @@ TEST(CacheKey, LanguageAndDialectChangeTheKey) { { auto b = axes(); b.dialectFlags = {"-freflection"}; EXPECT_NE(ck::key_hex(b, pkg()), base); } { auto b = axes(); b.cStandard = "c17"; EXPECT_NE(ck::key_hex(b, pkg()), base); } - { auto b = axes(); b.macosDeploymentTarget = "14.0"; + // ONE SLOT, BOTH PLATFORMS. It was `macosDeploymentTarget`; Android's + // minimum API level is the same quantity and is in this key for the same + // reason -- it selects which bionic symbols are visible, so two levels are + // two ABIs. A target is either Apple or Android, so one slot cannot be + // asked to hold both at once. + { auto b = axes(); b.minPlatformVersion = "14.0"; EXPECT_NE(ck::key_hex(b, pkg()), base); } + { auto b = axes(); b.minPlatformVersion = "24"; + EXPECT_NE(ck::key_hex(b, pkg()), base); } + // AND TWO LEVELS ARE TWO KEYS, which is the property the Android row + // depends on: without it `min_api_level = 21` and `= 24` would share a + // build directory and the second build would serve the first's objects. + { auto b21 = axes(); b21.minPlatformVersion = "21"; + auto b24 = axes(); b24.minPlatformVersion = "24"; + EXPECT_NE(ck::key_hex(b21, pkg()), ck::key_hex(b24, pkg())); } } // ── D axis: package identity ───────────────────────────────────────────────── diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 83ede0dc4..a2b2f44d9 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -645,6 +645,13 @@ TEST(Distribution, FormatIsTakenFromTheTargetAndNotTheFallback) { for (auto fb : {dist::Format::Elf, dist::Format::MachO, dist::Format::Pe}) { EXPECT_EQ(dist::format_for("aarch64-macos", fb), dist::Format::MachO); EXPECT_EQ(dist::format_for("x86_64-macos", fb), dist::Format::MachO); + // iOS shares macOS's Mach-O format. Before `format_for` asked + // `is_mach_o()` instead of `os == "macos"`, this triple matched none + // of the branches inside the parsed-triple block and fell through to + // `fb` itself -- so the assertion below would have failed for two of + // the three fallbacks in this loop, the exact "decided by the + // machine doing the building" defect this test exists to catch. + EXPECT_EQ(dist::format_for("aarch64-ios", fb), dist::Format::MachO); EXPECT_EQ(dist::format_for("x86_64-windows-gnu", fb), dist::Format::Pe); EXPECT_EQ(dist::format_for("x86_64-linux-gnu", fb), dist::Format::Elf); EXPECT_EQ(dist::format_for("aarch64-linux-musl", fb), dist::Format::Elf); @@ -739,3 +746,44 @@ TEST(Distribution, ASharedLibraryStillHidesTheArchivesWithoutASecondRuntime) { " -Wl,--exclude-libs,libc++.a -Wl,--exclude-libs,libc++abi.a" " /tc/libunwind.a -Wl,--exclude-libs,libunwind.a"); } + +// ─── Format::Wasm, the member the module predicted and deferred ──────────── +// +// `format_for`'s own comment said a wasm triple "falls out of every branch" +// and reaches `hostFallback`, answering the MACHINE's format -- "the same +// defect class its own header measured for macOS" -- and deferred the fourth +// member "to whoever gives this module a mechanism for it". +TEST(Distribution, FormatAnswersWasmForAWasmTargetAndNotTheHosts) { + using mcpp::build::dist::Format; + using mcpp::build::dist::format_for; + // Every host fallback, so this cannot pass by agreeing with the machine. + for (auto fallback : {Format::Elf, Format::MachO, Format::Pe}) + EXPECT_EQ(format_for("wasm32-emscripten", fallback), Format::Wasm); + // The three it already answered stay unchanged. + EXPECT_EQ(format_for("x86_64-linux-gnu", Format::Pe), Format::Elf); + EXPECT_EQ(format_for("aarch64-macos", Format::Elf), Format::MachO); + EXPECT_EQ(format_for("x86_64-windows-gnu", Format::Elf), Format::Pe); +} + +// THERE IS NOTHING TO BE COUPLED TO, so the contract is satisfied with +// nothing added and there is no degradation to report. +// +// While the member was missing, every wasm build printed a warning about a +// `libc++.so` that cannot exist for the target, on an artefact that has no +// run-time dependency of any kind. A diagnostic is for a BROKEN PROMISE; that +// one was a promise about a mechanism the format does not have. +TEST(Distribution, WasmIsSelfContainedByConstructionAndSaysNothing) { + mcpp::build::dist::MechanismInput in; + in.format = mcpp::build::dist::Format::Wasm; + in.stdlibId = "libc++"; + in.requested = mcpp::build::dist::Contract::SelfContained; + in.role = mcpp::build::dist::Role::Distributable; + auto m = mcpp::build::dist::resolve(in); + EXPECT_EQ(m.effective, mcpp::build::dist::Contract::SelfContained); + EXPECT_FALSE(m.degraded); + EXPECT_TRUE(m.diagnostic.empty()) << m.diagnostic; + // And no flags: libc++ reaches a wasm link through `em++`'s own link line, + // so naming archives from a sysroot this module did not resolve would be a + // second answer to a question the driver has already answered. + EXPECT_TRUE(m.unitFlags.empty()) << m.unitFlags; +} diff --git a/tests/unit/test_graph_shape.cpp b/tests/unit/test_graph_shape.cpp index 40a75ae9d..88c63f554 100644 --- a/tests/unit/test_graph_shape.cpp +++ b/tests/unit/test_graph_shape.cpp @@ -3,10 +3,21 @@ import std; import mcpp.build.graph_shape; // The header line build.ninja carries is what the fast paths read before any -// plan exists. Three facts ride it -- the graph's shape, the module-edge -// schedule, and (2026.9.5.3+) whether a `--accel` / `--no-accel` override -// chose the device variant -- and the fast paths decline a graph that says -// anything other than "plain build, manifest's own variant". +// plan exists. Four facts ride it -- the graph's shape, the module-edge +// schedule, (2026.9.5.3+) whether a `--accel` / `--no-accel` override chose +// the device variant, and (2026.9.11.1+) which distribution format the graph +// was generated for -- and the fast paths decline a graph that says anything +// other than "plain build, manifest's own variant, no distribution edge". +// +// THE FOURTH FIELD IS VERIFIED HERE AND NOT ONLY END TO END, deliberately. +// `mcpp pack --format ` prepares twice and the second pass writes its +// graph into the SAME fingerprint directory a plain build uses, because the +// format is not in the fingerprint. Measured on 2026年09月11日: a plain build +// after that pass regenerates the graph even with this field ignored, so some +// earlier freshness condition already declines -- which means an end-to-end +// assertion would pass whether or not the field works, and would keep passing +// if the field were deleted. A read-side invariant only has to hold once +// (see the module header); this is where it is held. namespace { @@ -26,11 +37,16 @@ std::filesystem::path write_graph(const std::string& first) { } // namespace -TEST(GraphShape, TheHeaderNamesShapeScheduleAndSelection) { +TEST(GraphShape, TheHeaderNamesShapeScheduleSelectionAndFormat) { EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false), - "# mcpp:graph=normal;schedule=none;accel=default"); + "# mcpp:graph=normal;schedule=none;accel=default;dist=none"); EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::WithTests, "two-phase", true), - "# mcpp:graph=test;schedule=two-phase;accel=override"); + "# mcpp:graph=test;schedule=two-phase;accel=override;dist=none"); + // An empty format reads as "none" rather than as an empty field: the value + // has to be a word, because `read_pack_format` returning "" already means + // "this file predates the field", and the two must not collide. + EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false, "appimage"), + "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage"); } TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { @@ -40,6 +56,19 @@ TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { EXPECT_FALSE(is_plain_build_graph(write_graph(header_line(GraphShape::Normal, "none", true)))); // The test-shaped graph was already refused. EXPECT_FALSE(is_plain_build_graph(write_graph(header_line(GraphShape::WithTests, "none", false)))); + // A DISTRIBUTION EDGE IS NOT PART OF A PLAIN BUILD. `mcpp pack --format + // appimage` makes a build program submit an artifact action consuming the + // staged tree; replaying that graph for a plain build would produce a + // distributable as a side effect of `mcpp build`, from a staged tree that + // is no longer guaranteed to describe this build. + EXPECT_FALSE(is_plain_build_graph( + write_graph(header_line(GraphShape::Normal, "none", false, "appimage")))); + EXPECT_EQ(read_pack_format( + write_graph(header_line(GraphShape::Normal, "none", false, "appimage"))), + "appimage"); + EXPECT_EQ(read_pack_format( + write_graph(header_line(GraphShape::Normal, "none", false))), + "none"); } TEST(GraphShape, AGraphThatPredatesTheFieldIsAMiss) { @@ -50,5 +79,15 @@ TEST(GraphShape, AGraphThatPredatesTheFieldIsAMiss) { auto p = write_graph("# mcpp:graph=normal;schedule=none"); EXPECT_EQ(read_shape(p), GraphShape::Normal); EXPECT_EQ(read_accel_selection(p), ""); + EXPECT_EQ(read_pack_format(p), ""); EXPECT_FALSE(is_plain_build_graph(p)); + + // Written by a 2026年9月10日.2 mcpp: shape, schedule and selection, no + // distribution field. Same rule one field later -- absent is a miss, and + // must not be read as "none", or the very first build after an upgrade + // would replay a graph this binary cannot describe. + auto q = write_graph("# mcpp:graph=normal;schedule=none;accel=default"); + EXPECT_EQ(read_accel_selection(q), "default"); + EXPECT_EQ(read_pack_format(q), ""); + EXPECT_FALSE(is_plain_build_graph(q)); } diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp index 34a9d7188..b9535c5d7 100644 --- a/tests/unit/test_hostflags.cpp +++ b/tests/unit/test_hostflags.cpp @@ -181,6 +181,62 @@ TEST(HostFlags, LanguageForceTokensNeverContainASpace) { } } +// ── orphaned_reference (#604) ─────────────────────────────────────────────── +// +// THE PAIR IS THE UNIT, and every per-token check passes while it is broken. +// MSVC's reference is two argv elements; a per-token de-duplicator dropped the +// second `/reference` -- already in the list from the bundled `mcpp` module -- +// and left `=` standing alone. cl.exe read it as a source file: +// +// c1xx: fatal error C1083: Cannot open source file: +// 'huxerui.rules.sources=...\huxerui.rules.sources.ifc' +// +// The de-duplicator is gone. This is the invariant that says so, and the +// diagnostic that would name the cause if it ever returns. +TEST(HostFlags, AnOrphanedModuleReferenceIsDetected) { + using mcpp::toolchain::orphaned_reference; + + // The defect, verbatim: two references appended, one switch surviving. + EXPECT_EQ(orphaned_reference({"cl.exe", "/std:c++20", "/reference", + "mcpp=C:/b/mcpp.ifc", + "rules.sources=C:/b/rules.sources.ifc", + "/c", "build.mcpp"}), + std::optional{"rules.sources=C:/b/rules.sources.ifc"}); + + // The same argv with both switches present is well formed. + EXPECT_FALSE(orphaned_reference({"cl.exe", "/std:c++20", "/reference", + "mcpp=C:/b/mcpp.ifc", "/reference", + "rules.sources=C:/b/rules.sources.ifc", + "/c", "build.mcpp"}).has_value()); + + // First position is an orphan too -- there is nothing in front of it. + EXPECT_TRUE(orphaned_reference({"mcpp=C:/b/mcpp.ifc"}).has_value()); +} + +// The rule must not fire on the two bare tokens that legitimately appear. +TEST(HostFlags, OrphanRuleAcceptsInputPathsAndOneWordReferences) { + using mcpp::toolchain::orphaned_reference; + + // Clang's form is ONE token and carries its own switch, so it can never be + // orphaned -- which is exactly why the defect was MSVC-only. + EXPECT_FALSE(orphaned_reference( + {"clang++", "-fmodule-file=mcpp=/b/mcpp.pcm", + "-fmodule-file=rules.sources=/b/rules.sources.pcm", + "-c", "build.mcpp"}).has_value()); + + // GCC names nothing at all. + EXPECT_FALSE(orphaned_reference( + {"g++", "-fmodules", "-fmodules", "-c", "build.mcpp"}).has_value()); + + // A source or object path that happens to contain `=` is an input, not a + // reference: its `=` comes AFTER a directory separator. Without this the + // rule would refuse a legal build in a directory a user is allowed to name. + EXPECT_FALSE(orphaned_reference( + {"g++", "/home/me/a=b/build.mcpp"}).has_value()); + EXPECT_FALSE(orphaned_reference( + {"cl.exe", "C:/b/x=y/build.mcpp"}).has_value()); +} + TEST(HostFlags, BmiReferenceIsEmptyForAToolchainThatNamesNothing) { // GCC finds BMIs implicitly under /gcm.cache — its prefix is empty // and must not produce a stray token. @@ -342,6 +398,18 @@ TEST(GraphRuntimeFlags, MachOTakesEmulatedTlsAndHiddenVisibilityButNotDwarf) { EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); } +// iOS shares macOS's object format (Mach-O, ld64), so it must share this +// exact set of flags. Before `is_mach_o()` replaced `os == "macos"` here, an +// iOS triple matched neither the PE nor the macOS branch and this function +// silently returned no flags at all for it. +TEST(GraphRuntimeFlags, IosTakesTheSameFlagsAsMacOS) { + auto f = mcpp::toolchain::graph_runtime_compile_flags(graph_tc("aarch64-ios")); + EXPECT_FALSE(has(f, "-fdwarf-exceptions")); + EXPECT_TRUE(has(f, "-femulated-tls")); + EXPECT_TRUE(has(f, "-fvisibility=hidden")); + EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); +} + // ELF takes NONE of them, and that is a decision rather than an omission. // There a `thread_local` is a fixed offset from the thread pointer, which the // C library establishes itself; adding the flag would work, cost an @@ -439,3 +507,113 @@ TEST(HostFlags, TheCfgBypassSurvivesAGraphSuppliedTargetSide) { EXPECT_TRUE(has(a, "-nostdinc++")); EXPECT_FALSE(has(b, "-nostdinc++")); } + +// "A TOOLCHAIN THAT SHIPS ITS OWN SYSROOT IS TOLD NOTHING" WAS ONE TOKEN TOO +// STRONG, AND THIS FUNCTION ALREADY SAID SO FURTHER DOWN. +// +// The early return for `has_own_sysroot()` withholds the target's system +// reconstructed onto the command line -- libc++'s headers, glibc's, the Linux +// UAPI headers, the cfg bypass, the C-runtime prefix -- because an Emscripten +// or Android SDK already has all of it. That is right. It stood in FRONT of +// the paragraph beginning "THE TRIPLE, SAID OUT LOUD", which states the +// opposite rule for the same underlying reason: an ordinary clang emits for the +// machine it is running on unless told otherwise. The stronger claim won by +// position. +// +// Both are right about their own object. The SYSTEM is the payload's; WHICH +// TARGET is still mcpp's to say, because one NDK serves both Android ABIs and +// nothing else on the command line distinguishes them. The defect was reported +// by neither compile but by the module loader: +// +// error: AST file 'std.pcm' was compiled for the target +// 'aarch64-unknown-linux-android21' but the current translation unit is +// being compiled for target 'x86_64-unknown-linux-gnu' +// +// followed by eight cascading "use of undeclared identifier 'std'" lines, +// which is what a reader sees first. +TEST(HostFlags, AnOwnSysrootTargetIsToldWhichTargetAndNothingElse) { + HostFlagOptions opt; + + for (auto name : {"aarch64-linux-android", "x86_64-linux-android", + "wasm32-emscripten"}) { + auto tc = tc_for(CompilerId::Clang); + tc.targetTriple = name; + tc.crossTargetFlag = "--target=SENTINEL-TRIPLE"; + + auto tokens = mcpp::toolchain::host_compile_tokens( + tc, opt, mcpp::toolchain::no_escape); + + // EXACTLY the target flag. Asserted as the whole vector rather than as + // "contains", because the property is that nothing ELSE is emitted: + // this host's glibc headers reaching a wasm compile is the measured + // failure this gate exists for. + ASSERT_EQ(tokens.size(), 1u) << name << ": " << [&] { + std::string all; + for (auto const& t : tokens) { all += t; all += ' '; } + return all; + }(); + EXPECT_EQ(tokens[0], "-- << name; + } + + // AND THE GATE IS STILL A GATE, DISCRIMINATED BY THE cfg BYPASS. + // + // A first version of this control asserted that a HOSTED target receives + // more than one token, and it failed -- with a bare `Toolchain` carrying no + // payload paths, the hosted path has nothing to reconstruct either, so both + // sides produced exactly the triple and the control could not tell them + // apart. The control was wrong, not the code. + // + // `--no-default-config` is the discriminator, and it is a property the + // gate's own comment states: the bypass exists to stop clang reading a + // per-install `clang++.cfg`, while `em++` is a wrapper whose entire job is + // to supply configuration, so suppressing it would be suppressing the + // toolchain. It is therefore emitted past the gate and never before it, + // which is exactly what a control needs. + // A first version of this control asserted only that a HOSTED target + // receives more than one token, and it failed -- with a bare `Toolchain` + // carrying no payload the hosted path has nothing to reconstruct either, + // so both sides produced exactly the triple and the control could not tell + // them apart. A second version reached for `--no-default-config` without a + // payload that HAS a cfg, which is the same mistake once removed. The + // fixture is what makes the discriminator real. + // + // `--no-default-config` is the right discriminator because it is a + // property the gate's own comment states: the bypass exists to stop clang + // reading a per-install `clang++.cfg`, while `em++` is a wrapper whose + // entire job is to supply configuration, so suppressing it would be + // suppressing the toolchain. Emitted past the gate, never before it. + FakeClangPayload payload{"own-sysroot-gate"}; + HostFlagOptions bypass; + bypass.cfgBypass = HostFlagOptions::CfgBypass::Always; + + auto host = tc_for(CompilerId::Clang); + host.binaryPath = payload.root / "bin" / "clang++"; + host.crossTargetFlag = "-- + auto hostTokens = mcpp::toolchain::host_compile_tokens( + host, bypass, mcpp::toolchain::no_escape); + EXPECT_NE(std::ranges::find(hostTokens, "--no-default-config"), + hostTokens.end()) + << "a hosted clang with a cfg beside it must reach the bypass"; + + for (auto name : {"aarch64-linux-android", "wasm32-emscripten"}) { + auto sdk = tc_for(CompilerId::Clang); + sdk.binaryPath = payload.root / "bin" / "clang++"; // same payload + sdk.targetTriple = name; + sdk.crossTargetFlag = "-- + auto sdkTokens = mcpp::toolchain::host_compile_tokens( + sdk, bypass, mcpp::toolchain::no_escape); + EXPECT_EQ(std::ranges::find(sdkTokens, "--no-default-config"), + sdkTokens.end()) + << name << ": the cfg bypass must be withheld from an SDK whose " + "driver's job is to supply configuration"; + EXPECT_EQ(sdkTokens.size(), 1u) << name; + } + + // A row with no cross flag emits nothing at all rather than an empty + // token: an empty argv element is an argument the driver must interpret. + auto bare = tc_for(CompilerId::Clang); + bare.targetTriple = "wasm32-emscripten"; + ASSERT_TRUE(bare.crossTargetFlag.empty()); + EXPECT_TRUE(mcpp::toolchain::host_compile_tokens( + bare, opt, mcpp::toolchain::no_escape).empty()); +} diff --git a/tests/unit/test_linkmodel.cpp b/tests/unit/test_linkmodel.cpp index a4f3b5c20..76354229e 100644 --- a/tests/unit/test_linkmodel.cpp +++ b/tests/unit/test_linkmodel.cpp @@ -237,3 +237,31 @@ TEST(LinkModel, NothingUsableYieldsNoneAndEmptyFlags) { // 设计:.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md §3.2 } // namespace + +// ─── An SDK target gets CLibMode::None, and the gate is in the MODEL ─────── +// +// The C-runtime group reaches the link line through TWO channels -- flags.cppm's +// `link_toolchain_flags` and its `payload_ld` -- both rendering +// `lm.link_flags()`. The comment at the second one records that a reader who +// fixed only the first "saw the identical error and could reasonably conclude +// the fix had not worked". So the gate belongs here, where both read it. +// +// Measured on `--target wasm32-emscripten` with the compile side already +// correct: `wasm-ld: error: unknown argument: +// --dynamic-linker=/lib64/ld-linux-x86-64.so.2` -- this host's loader +// handed to a WebAssembly linker. +TEST(LinkModel, AnSdkTargetDescribesNoCLibrary) { + for (auto target : {"wasm32-emscripten", "aarch64-linux-android", + "x86_64-linux-android"}) { + auto tc = mcpp::toolchain::Toolchain{}; + tc.compiler = mcpp::toolchain::CompilerId::Clang; + tc.targetTriple = target; + auto lm = mcpp::toolchain::resolve_link_model(tc); + EXPECT_EQ(lm.mode, mcpp::toolchain::CLibMode::None) << target; + EXPECT_TRUE(lm.libDirs.empty()) << target; + EXPECT_TRUE(lm.crtDir.empty()) << target; + // And the rendered flags are empty, which is what both channels emit. + EXPECT_TRUE(lm.link_flags([](const std::filesystem::path& p) { + return p.string(); }).empty()) << target; + } +} diff --git a/tests/unit/test_loader_contract.cpp b/tests/unit/test_loader_contract.cpp index 52643e75a..1862b219d 100644 --- a/tests/unit/test_loader_contract.cpp +++ b/tests/unit/test_loader_contract.cpp @@ -60,9 +60,18 @@ TEST(GraphShape, UnlabelledOrUnknownGraphIsNeverPlain) { }; EXPECT_TRUE(is_plain_build_graph( - write("normal.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default\nrule x\n"))); + write("normal.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default;dist=none\nrule x\n"))); EXPECT_FALSE(is_plain_build_graph( - write("test.ninja", "# banner\n# mcpp:graph=test;schedule=none;accel=default\nrule x\n"))); + write("test.ninja", "# banner\n# mcpp:graph=test;schedule=none;accel=default;dist=none\nrule x\n"))); + + // A plain-shaped graph that `mcpp pack --format ` wrote (2026年9月11日.1+): + // it carries an artifact edge consuming a staged tree, which a plain build + // must not have. These lines are spelled by hand here rather than through + // `header_line`, which is the point of this copy -- a reader of build.ninja + // sees the text, and a field added to the producer without being added to + // the reader would still pass a test that only compared the two. + EXPECT_FALSE(is_plain_build_graph( + write("dist.ninja", "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage\n"))); // A plain-shaped graph an `--accel` / `--no-accel` build wrote (2026年9月5日.3+): // the variant a flag chose is not the variant a plain build produces. @@ -74,6 +83,14 @@ TEST(GraphShape, UnlabelledOrUnknownGraphIsNeverPlain) { EXPECT_FALSE(is_plain_build_graph( write("no-selection.ninja", "# banner\n# mcpp:graph=normal\nrule x\n"))); + // A graph from 2026年9月10日.2: shape, schedule and selection, no distribution + // field. Same rule one field later. This assertion is what caught the + // second copy of these lines when the field was added -- the line above it + // was the CURRENT spelling in one file and became the LEGACY spelling in + // both, and only a test that spells it out could say so. + EXPECT_FALSE(is_plain_build_graph( + write("no-dist.ninja", "# banner\n# mcpp:graph=normal;schedule=none;accel=default\nrule x\n"))); + // A build.ninja from before the marker existed. It MUST read as a miss: // treating it as plain is precisely the replay #407 is about. EXPECT_FALSE(is_plain_build_graph( diff --git a/tests/unit/test_pack_relocate.cpp b/tests/unit/test_pack_relocate.cpp index 518819287..c1146ea70 100644 --- a/tests/unit/test_pack_relocate.cpp +++ b/tests/unit/test_pack_relocate.cpp @@ -325,6 +325,8 @@ TEST(PackStrip, WhetherStrippingAppliesIsAskedOfTheTargetNotTheCompiler) { EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("x86_64-windows-msvc")); EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("aarch64-macos")); EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("x86_64-macos")); + // iOS carries the same Mach-O debug map + out-of-band .dSYM as macOS. + EXPECT_FALSE(mcpp::pack::debug_info_is_in_band("aarch64-ios")); // Segment-wise, not substring: mcpp has been bitten by a triple predicate // that answered on a substring before. EXPECT_TRUE(mcpp::pack::debug_info_is_in_band("macos64-linux-gnu")); diff --git a/tests/unit/test_pack_stage_tree.cpp b/tests/unit/test_pack_stage_tree.cpp new file mode 100644 index 000000000..5bf709694 --- /dev/null +++ b/tests/unit/test_pack_stage_tree.cpp @@ -0,0 +1,141 @@ +#include + +import std; +import mcpp.pack.stage_tree; + +// The staged tree is what `mcpp pack` computes and, until `${mcpp.stage_dir}`, +// then threw away. Two things about it are the engine's contract with a +// distribution member, and both are asserted here rather than end to end, +// because the end-to-end criterion for either is "the distributable is +// rebuilt", which a wrong answer also satisfies. + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_stage_tree_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +void write_file(const std::filesystem::path& p, std::string_view body) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream os(p, std::ios::binary); + os << body; +} + +std::string read_file(const std::filesystem::path& p) { + std::ifstream is(p, std::ios::binary); + return std::string{std::istreambuf_iterator(is), {}}; +} + +} // namespace + +TEST(PackStageTree, TheManifestIsASiblingAndNeverAMember) { + Tmp t; + auto stage = t.path / "app-1.0.0-x86_64-linux-gnu"; + std::filesystem::create_directories(stage); + + auto manifest = mcpp::pack::stage_manifest_path(stage); + EXPECT_EQ(manifest.parent_path(), stage.parent_path()); + // A file INSIDE the tree would be collected by every format that packages + // the directory wholesale, and would then ship inside the user's + // installer. Asserted as a path relationship because that is the property, + // and a spelling change that broke it would otherwise only show up as an + // extra file in a released package. + EXPECT_FALSE(manifest.string().starts_with(stage.string() + "/")); + EXPECT_FALSE(manifest.string().starts_with(stage.string() + "\\")); + + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + std::error_code ec; + for (auto const& e : std::filesystem::recursive_directory_iterator(stage, ec)) + FAIL() << "the manifest landed inside the tree: " << e.path().string(); +} + +TEST(PackStageTree, TheManifestChangesWhenTheStagedSetDoes) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "0123456789"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + const auto first = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(first.find("bin/app"), std::string::npos); + EXPECT_NE(first.find("10 "), std::string::npos); + + // A DEPENDENCY'S SHARED LIBRARY JOINING THE CLOSURE. This is the case the + // manifest exists for: the program's own bytes need not have changed, so an + // edge that depended only on the link output would report the previous + // distributable as up to date. + write_file(stage / "lib" / "libdep.so.1", "xx"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + const auto second = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(first, second); + EXPECT_NE(second.find("lib/libdep.so.1"), std::string::npos); + + // A staged file whose LENGTH changed, with no entry added or removed. + write_file(stage / "bin" / "app", "0123456789abcdef"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + EXPECT_NE(read_file(mcpp::pack::stage_manifest_path(stage)), second); +} + +TEST(PackStageTree, StagingTheSameTreeTwiceLeavesTheManifestUntouched) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "same"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + auto manifest = mcpp::pack::stage_manifest_path(stage); + const auto before = std::filesystem::last_write_time(manifest); + + // Rewriting identical bytes would move the mtime, and a moved mtime on an + // input is indistinguishable from a changed input: a pack that staged the + // same tree twice would rebuild the distributable both times. Same rule + // `mcpp.build.stage` states at length for a staged BMI. + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + EXPECT_EQ(std::filesystem::last_write_time(manifest), before); +} + +TEST(PackStageTree, TheOrderOfADirectoryWalkIsNotAPromise) { + // Two trees with the same contents produce the same bytes. Without the + // sort, an iteration order that differed between runs would make the dist + // edge dirty on every pack for no reason -- which reads as "packaging is + // slow" rather than as a defect. + Tmp a, b; + for (auto const& root : {a.path, b.path}) { + write_file(root / "s" / "bin" / "app", "aa"); + write_file(root / "s" / "lib" / "z.so", "bbb"); + write_file(root / "s" / "share" / "doc" / "readme", "c"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(root / "s")); + } + EXPECT_EQ(read_file(mcpp::pack::stage_manifest_path(a.path / "s")), + read_file(mcpp::pack::stage_manifest_path(b.path / "s"))); +} + +TEST(PackStageTree, TheEngineOwnsExactlyTwoFormatNames) { + // `tar` and `dir` are the archive shapes `mcpp pack` owns; every other + // value of `--format` is a name a package provides. The list lives beside + // the staged tree because two layers need the same answer -- the parser, + // which decides whether a value is a built-in or a dispatch, and the + // declaration check, which refuses a package that claims one of them and + // would therefore be silently unreachable. + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("tar")); + EXPECT_TRUE(mcpp::pack::is_builtin_pack_format("dir")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("appimage")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("msi")); + EXPECT_FALSE(mcpp::pack::is_builtin_pack_format("")); + EXPECT_EQ(mcpp::pack::kBuiltinPackFormats.size(), 2u); +} + +TEST(PackStageTree, AMissingTreeIsRefusedRatherThanDescribedAsEmpty) { + Tmp t; + // An empty manifest for a directory that does not exist would say "nothing + // is staged", which is what a correct pack of an empty bundle also says. + // The two must not be spelled alike. + EXPECT_FALSE(mcpp::pack::write_stage_manifest(t.path / "never-staged")); + EXPECT_FALSE(std::filesystem::exists( + mcpp::pack::stage_manifest_path(t.path / "never-staged"))); +} diff --git a/tests/unit/test_schedule_policy.cpp b/tests/unit/test_schedule_policy.cpp index 496e3cdf7..c0c1abdba 100644 --- a/tests/unit/test_schedule_policy.cpp +++ b/tests/unit/test_schedule_policy.cpp @@ -170,6 +170,52 @@ TEST(SchedulePolicy, AnAbsurdJobCountDoesNotOverflowIntoANegativeOne) { EXPECT_GT(d.ninjaJobs, 1) << "hazard 2: ninja must still outnumber the compilers"; } +// ─── resolve_jobs precedence (#564) ──────────────────────────────────────── +// +// A KEY THE GENERATED FILE PROMISED AND NOTHING READ. mcpp writes +// `[build] default_jobs = 0` into `$MCPP_HOME/config.toml` itself; the loader +// parsed it into `GlobalConfig::defaultJobs`, and those were the only two +// mentions of the field in the repository. Setting it to 4 changed nothing: +// `ninja` ran with its own default, which is 10 on an 8-core machine, while a +// single module compile peaks at 0.5-1.0 GB. +// +// THE TEST ASSERTS THE ORDER, NOT THE WIRING. A fixture that sets only the +// global value and reads it back would pass just as well if the parameter had +// been placed ABOVE `MCPP_JOBS` instead of below it -- which would be the +// opposite of correct, because the global value is a property of the machine +// and the environment variable is this invocation. Each of the three levels is +// therefore checked against the level that must beat it. +TEST(SchedulePolicy, ResolveJobsPutsTheMachineBelowTheProjectAndTheInvocation) { + mcpp::manifest::Manifest bare; // no [build] jobs + auto with_jobs = [](std::string v) { + mcpp::manifest::Manifest m; + m.buildConfig.jobs = std::move(v); + return m; + }; + using mcpp::build::schedule::resolve_jobs; + + { // The machine's value is used when nothing else says anything, and 0 + // still means "say nothing" for everyone who has not written the key. + ScopedVar clear("MCPP_JOBS", nullptr); + EXPECT_EQ(resolve_jobs(bare, {}, 4), 4); + EXPECT_EQ(resolve_jobs(bare, {}, 0), 0); + // A non-positive value reads as absent rather than as a bound: that is + // what the generated template's `0` means, and a negative -j would be + // handed straight to the backend. + EXPECT_EQ(resolve_jobs(bare, {}, -1), 0); + + // The project beats the machine. A project that states a number has a + // reason the machine cannot know. + EXPECT_EQ(resolve_jobs(with_jobs("6"), {}, 4), 6); + } + { // The invocation beats both. Without this leg the parameter could be + // wired in above MCPP_JOBS and every other assertion here would pass. + ScopedVar jobs("MCPP_JOBS", "2"); + EXPECT_EQ(resolve_jobs(bare, {}, 4), 2); + EXPECT_EQ(resolve_jobs(with_jobs("6"), {}, 4), 2); + } +} + // ─── requested_switch: a typo is a diagnostic, never a silent "auto" ─────── // // This is the rule resolve_jobs already followed and this switch did not. diff --git a/tests/unit/test_toolchain_msvc.cpp b/tests/unit/test_toolchain_msvc.cpp index a49680235..b6ca5a857 100644 --- a/tests/unit/test_toolchain_msvc.cpp +++ b/tests/unit/test_toolchain_msvc.cpp @@ -302,7 +302,12 @@ TEST(MsvcManaged, PayloadFrontendFindsClWhereMsvcActuallyKeepsIt) { ASSERT_TRUE(spec.has_value()); auto pkg = to_xim_package(*spec); - auto found = payload_frontend(t.root, pkg, Family::Msvc); + // The family comes from the PACKAGE now, not from a second argument every + // caller had to source from a differently-named local -- five of them + // composed `payload->binDir` themselves instead and so could not see + // `frontendSubdir` at all. + EXPECT_EQ(pkg.family, Family::Msvc); + auto found = payload_frontend(t.root, pkg); ASSERT_FALSE(found.empty()) << "payload_frontend found no cl.exe under " << t.root; EXPECT_EQ(found.filename(), "cl.exe"); @@ -314,8 +319,16 @@ TEST(MsvcManaged, PayloadFrontendFindsClWhereMsvcActuallyKeepsIt) { // A root with no toolset at that version stays empty rather than // returning a path that does not exist. EXPECT_TRUE(payload_frontend(t.root, - to_xim_package(*parse_toolchain_spec("msvc@14.52.36629")), - Family::Msvc).empty()); + to_xim_package(*parse_toolchain_spec("msvc@14.52.36629"))) + .empty()); + + // AND THE DIRECTORY THE SEARCH USED IS AVAILABLE FOR THE MESSAGE. MSVC is + // the case that proves it is a lookup rather than a constant: the path + // carries the toolset version, so a refusal naming `bin` would name a + // directory nothing looked in. + auto dir = payload_frontend_dir(t.root, pkg); + EXPECT_FALSE(dir.empty()); + EXPECT_NE(dir.string().find("14.44.35207"), std::string::npos) << dir; } // ─── Windows SDK discovery ─────────────────────────────────────────────── @@ -574,6 +587,59 @@ TEST(MsvcStdModule, MinLevelFollowsStlUnblockVersion) { EXPECT_EQ(msvc::std_module_min_level(tc_of("unknown")), 23); } +// #603 — THE QUESTION IS ASKED OF THE STL, NOT OF WHATEVER COMPILER REACHES IT. +// +// Two compilers reach the same `std.ixx`: cl.exe under `msvc@system`, and clang +// targeting `*-windows-msvc` with no libc++ std module present. The clang path +// used to hardcode 23, with a comment that named the reason correctly -- +// `tc.version` is clang's there -- and drew the wrong conclusion from it. +// +// Calling the banner form from that path would be worse than the hardcode, +// because it would compare a CLANG version number against an MSVC threshold and +// be right by accident: clang 20.x passes `>= 19.38` and clang 19.x fails it. +// The toolset version is in the path of the module source that was selected. +TEST(MsvcStdModule, MinLevelForStlReadsTheToolsetOutOfTheModuleSourcePath) { + auto lvl = [](std::string p) { + return msvc::std_module_min_level_for_stl(std::filesystem::path(p)); + }; + // VS 2022 17.8 (toolset 14.38) is where microsoft/STL#3977 first ships. + EXPECT_EQ(lvl("C:/VS/VC/Tools/MSVC/14.38.33130/modules/std.ixx"), 20); + EXPECT_EQ(lvl("C:/VS/VC/Tools/MSVC/14.44.35207/modules/std.ixx"), 20); + // Older toolsets keep the C++23 floor and get an actionable diagnostic. + EXPECT_EQ(lvl("C:/VS/VC/Tools/MSVC/14.37.32822/modules/std.ixx"), 23); + EXPECT_EQ(lvl("C:/VS/VC/Tools/MSVC/14.29.30133/modules/std.ixx"), 23); + // A layout this mapping is not defined for answers 23 rather than guessing. + // The safety the hardcode was after is kept for exactly these cases. + EXPECT_EQ(lvl(""), 23); + EXPECT_EQ(lvl("C:/VS/VC/Tools/MSVC/modules/std.ixx"), 23); + EXPECT_EQ(lvl("/opt/llvm/share/libc++/v1/std.cppm"), 23); + EXPECT_EQ(lvl("C:/VS/VC/Tools/MSVC/15.0.0/modules/std.ixx"), 23); +} + +// The two forms must agree for a well-formed installation, because that is the +// one path the banner form was verified on. If they could disagree there, this +// change would be a silent behaviour change on the toolchain that worked. +TEST(MsvcStdModule, TheStlAndBannerFormsAgreeForAWellFormedInstallation) { + auto tc_of = [](std::string ver) { + Toolchain tc; + tc.compiler = CompilerId::MSVC; + tc.version = std::move(ver); + return tc; + }; + // cl banner `19.` pairs with toolset `14.` -- that pairing is the + // whole reason the `>= 38` predicate transfers unchanged. + for (auto [banner, toolset] : { std::pair{"19.38.33130", "14.38.33130"}, + std::pair{"19.44.35211", "14.44.35207"}, + std::pair{"19.37.32825", "14.37.32822"}, + std::pair{"19.29.30153", "14.29.30133"} }) { + EXPECT_EQ(msvc::std_module_min_level(tc_of(banner)), + msvc::std_module_min_level_for_stl(std::filesystem::path( + std::string("C:/VS/VC/Tools/MSVC/") + toolset + + "/modules/std.ixx"))) + << "banner " << banner << " vs toolset " << toolset; + } +} + // #422 — the std module must be built with the SAME CRT model as the TUs that // import it. // diff --git a/tests/unit/test_toolchain_registry.cpp b/tests/unit/test_toolchain_registry.cpp index 2437d9689..512766d01 100644 --- a/tests/unit/test_toolchain_registry.cpp +++ b/tests/unit/test_toolchain_registry.cpp @@ -235,6 +235,68 @@ TEST(ToolchainRegistry, NativeGccPayloadFollowsWhatTheArchActuallyPublishes) { // ONE platform — Visual Studio is very often already installed and cannot // always be redistributed. +// THE NDK'S OWN FLOOR, READ FROM THE PAYLOAD RATHER THAN COMPILED IN. +// +// Android's API level is not optional -- bionic's stops the build +// with "Unversioned target triples are not supported!" -- so a project that +// declares no `min_api_level` still needs one. The number comes from +// `meta/platforms.json`, which is upstream's own declaration of the range it +// supports, so a newer NDK changes the default by being installed. +// +// A CONSTANT HERE WOULD BE THE DEFECT THIS AVOIDS: this repository has +// recorded more than once that a version written into a comment becomes a +// version in a diagnostic and then in somebody's install command. +TEST(ToolchainRegistry, TheNdkApiLevelFloorIsReadFromThePayloadsOwnMetadata) { + namespace fs = std::filesystem; + auto root = fs::temp_directory_path() + / ("mcpp-ndk-meta-" + std::to_string(::getpid())); + fs::remove_all(root); + // The real layout: the compiler sits four directories below the NDK root, + // and `meta/` is a sibling of `toolchains/`. + auto bin = root / "toolchains" / "llvm" / "prebuilt" / "linux-x86_64" / "bin"; + fs::create_directories(bin); + fs::create_directories(root / "meta"); + auto clangxx = bin / "clang++"; + { std::ofstream o(clangxx); o << "#!/bin/sh\n"; } + + // r30's actual values. + { + std::ofstream o(root / "meta" / "platforms.json"); + o << R"({"min": 21, "max": 37, "aliases": {"N": 24}})"; + } + EXPECT_EQ(mcpp::toolchain::ndk_min_api_level(clangxx), 21); + + // A DIFFERENT PAYLOAD ANSWERS DIFFERENTLY, which is the whole point of + // reading it: the same code must not return 21 for an NDK that says 24. + { + std::ofstream o(root / "meta" / "platforms.json"); + o << R"({"min": 24, "max": 40})"; + } + EXPECT_EQ(mcpp::toolchain::ndk_min_api_level(clangxx), 24); + + // 0 WHEN IT CANNOT BE READ, and the caller turns that into a refusal + // naming `min_api_level`. A guessed level would be worse than the refusal: + // it selects which bionic symbols exist, so guessing produces an artefact + // that links here and fails to load on a device. + fs::remove(root / "meta" / "platforms.json"); + EXPECT_EQ(mcpp::toolchain::ndk_min_api_level(clangxx), 0); + + // Malformed rather than absent -- same answer, and no exception escapes. + { std::ofstream o(root / "meta" / "platforms.json"); o << "{not json"; } + EXPECT_EQ(mcpp::toolchain::ndk_min_api_level(clangxx), 0); + + // Present but not a number: still 0, never a silent 1 from a cast. + { std::ofstream o(root / "meta" / "platforms.json"); o << R"({"min": "21"})"; } + EXPECT_EQ(mcpp::toolchain::ndk_min_api_level(clangxx), 0); + + // A path that is not inside an NDK at all walks to the filesystem root and + // stops; it must not loop. + EXPECT_EQ(mcpp::toolchain::ndk_min_api_level( + fs::temp_directory_path() / "definitely-not-an-ndk" / "clang++"), 0); + + fs::remove_all(root); +} + TEST(ToolchainOrigin, MsvcIsTheOnlyFamilyWithASystemSpelling) { auto msvcSystem = parse_toolchain_spec("msvc@system"); ASSERT_TRUE(msvcSystem.has_value()) << msvcSystem.error(); @@ -295,3 +357,138 @@ TEST(ToolchainSysrootDeps, OneDerivationForTheGlibcSysrootPayloads) { EXPECT_FALSE(needs_linux_sysroot_payloads(t)); } } + +// ─── An SDK payload is chosen by the TARGET, and knows its own layout ────── +// +// `to_xim_package` returned the generic llvm payload for every `Family::Llvm` +// spec, which is right for the targets llvm itself serves and wrong for the two +// that arrive with their own clang. `em++` and the NDK's `clang++` ARE clang -- +// same family, same flag vocabulary -- so no fourth `Family` value exists; +// what changes is which package answers and where its driver lives. +TEST(SdkPayloads, TheTargetChoosesThePackageAndThePackageKnowsItsLayout) { + auto pkg_for = [](std::string_view target) { + auto spec = mcpp::toolchain::parse_toolchain_spec("emsdk@6.0.9"); + // The spec's own target is replaced, because the NDK case must be + // reachable from the same family with a different triple. + auto s = *spec; + if (auto t = mcpp::toolchain::triple::parse(target)) s.target = *t; + return mcpp::toolchain::to_xim_package(s); + }; + + { // Emscripten: `em++` is a wrapper in `emscripten/`, NOT the raw clang + // in `bin/`. Naming the wrapper is the whole point -- `bin/clang` + // compiles for wasm and then links like an ordinary clang, producing a + // module with none of Emscripten's JavaScript glue. + auto pkg = pkg_for("wasm32-emscripten"); + EXPECT_EQ(pkg.ximName, "emsdk"); + EXPECT_EQ(pkg.frontendSubdir, "emscripten"); + ASSERT_FALSE(pkg.frontendCandidates.empty()); + EXPECT_EQ(pkg.frontendCandidates.front(), "em++"); + } + { // Android: ONE payload for both arches -- the arch arrives as + // `-->-linux-android`, not as a different package -- + // and the host tuple in the path is the HOST's, not the target's. + for (auto target : {"aarch64-linux-android", "x86_64-linux-android"}) { + auto pkg = pkg_for(target); + EXPECT_EQ(pkg.ximName, "android-ndk") << target; + EXPECT_NE(pkg.frontendSubdir.find("toolchains/llvm/prebuilt/"), + std::string::npos) << target; + // The HOST, so an aarch64 Linux machine cross-compiling still + // reads `linux-x86_64`. Asserted as "not the target's arch" rather + // than against a literal, so this test says the same thing on + // every runner. + EXPECT_EQ(pkg.frontendSubdir.find("aarch64-linux-android"), + std::string::npos) << target; + } + } + { // And every other target still gets the generic llvm payload in bin/. + auto pkg = pkg_for("x86_64-linux-gnu"); + EXPECT_NE(pkg.ximName, "emsdk"); + EXPECT_NE(pkg.ximName, "android-ndk"); + EXPECT_EQ(pkg.frontendSubdir, "bin"); + } +} + +// THE MESSAGE MUST NAME THE DIRECTORY THAT WAS SEARCHED. +// +// Five refusals printed `payload->binDir`, the directory they had composed +// themselves. Once the package decides where its frontend lives, `bin` is a +// directory nothing looked in -- and this codebase's most frequent defect is a +// fixed lookup with an unfixed message. +TEST(SdkPayloads, TheSearchedDirectoryIsAvailableForTheDiagnostic) { + auto spec = mcpp::toolchain::parse_toolchain_spec("emsdk@6.0.9"); + ASSERT_TRUE(spec.has_value()); + auto pkg = mcpp::toolchain::to_xim_package(*spec); + auto dir = mcpp::toolchain::payload_frontend_dir("/p/xim-x-emsdk/6.0.9", pkg); + EXPECT_EQ(dir, std::filesystem::path("/p/xim-x-emsdk/6.0.9/emscripten")); +} + +// AN SDK IS SERVED WHERE THE SDK IS PUBLISHED, and the first version of this +// said "wherever" -- the same over-broad shape as the branches it sits above. +// The target matrix caught it: declaring the row servable on macOS and Windows +// would have claimed a payload that does not exist there. +TEST(SdkPayloads, ServedOnEveryHostTheSdkIsPublishedFor) { + // THIS ASSERTION USED TO BE TRUE BY ARITHMETIC ON ONE HOST. + // + // It read `EXPECT_EQ(host_can_serve(*wasm), mcpp::platform::is_linux)`, + // which was the right claim while `xim:emsdk` and `xim:android-ndk` + // declared only `xpm.linux`. Both now publish for all three hosts, and the + // engine's constant was the stale half -- but the assertion kept passing + // on Linux, because there `is_linux` IS `true`. A criterion whose expected + // value is the host it runs on cannot report a change on the other two. + // + // Stated unconditionally now: these rows are servable everywhere, and this + // test fails on macOS or Windows if the constant comes back. + for (auto name : {"wasm32-emscripten", "aarch64-linux-android", + "x86_64-linux-android"}) { + auto t = mcpp::toolchain::triple::parse(name); + ASSERT_TRUE(t.has_value()) << name; + EXPECT_TRUE(mcpp::toolchain::host_can_serve(*t)) << name; + } + + // AND THE PREDICATE IS STILL ABLE TO SAY NO, which is what keeps the + // paragraph above from being a tautology. macOS's SDK and MSVC are + // host-only and no package substitutes for either, so a Linux host cannot + // serve them -- the exclusion this function exists to make. + if constexpr (mcpp::platform::is_linux) { + auto mac = mcpp::toolchain::triple::parse("aarch64-macos"); + ASSERT_TRUE(mac.has_value()); + EXPECT_FALSE(mcpp::toolchain::host_can_serve(*mac)); + } +} + +// ─── The payload is SAID, not only resolved (R3) ─────────────────────────── +// +// `emsdk@6.0.9` normalises to the llvm family because `em++` IS clang, and a +// fourth family value would be a false claim about the compiler. The +// consequence was `Resolved llvm@6.0.9` -- indistinguishable from the real +// `xim:llvm`, and not what the user typed. The family and the payload are two +// questions, and `to_xim_package` already answered the second; this is the +// field that lets it be printed. +TEST(SdkPayloads, TheDisplayNamesThePayloadAndNotOnlyTheFamily) { + auto em = mcpp::toolchain::parse_toolchain_spec("emsdk@6.0.9"); + ASSERT_TRUE(em.has_value()); + EXPECT_EQ(em->family, mcpp::toolchain::Family::Llvm) + << "em++ is clang; a fourth family would be a false claim"; + EXPECT_EQ(em->payloadName, "emsdk"); + EXPECT_NE(em->display().find("emsdk@6.0.9"), std::string::npos) + << em->display(); + EXPECT_EQ(em->display().find("llvm@"), std::string::npos) << em->display(); + + auto ndk = mcpp::toolchain::parse_toolchain_spec("android-ndk@30.0.16248370"); + ASSERT_TRUE(ndk.has_value()); + EXPECT_EQ(ndk->family, mcpp::toolchain::Family::Llvm); + EXPECT_EQ(ndk->payloadName, "android-ndk"); + EXPECT_NE(ndk->display().find("android-ndk@"), std::string::npos) + << ndk->display(); + + // AND NOTHING ELSE MOVES. Empty `payloadName` means the family's own + // payload, which is every row but these two -- so no existing output + // changes, which is what makes this additive. + for (auto spelled : {"llvm@22.1.8", "gcc@16.1.0", "msvc@14.44.35207"}) { + auto sp = mcpp::toolchain::parse_toolchain_spec(spelled); + ASSERT_TRUE(sp.has_value()) << spelled; + EXPECT_TRUE(sp->payloadName.empty()) << spelled; + EXPECT_NE(sp->display().find(spelled), std::string::npos) << sp->display(); + } +} diff --git a/tests/unit/test_toolchain_stdmod.cpp b/tests/unit/test_toolchain_stdmod.cpp index dc337b405..b7e41eaba 100644 --- a/tests/unit/test_toolchain_stdmod.cpp +++ b/tests/unit/test_toolchain_stdmod.cpp @@ -70,3 +70,72 @@ TEST(ToolchainStdmod, ClangStdCompatCommandsUseRequestedStandard) { EXPECT_EQ(cmd.find("-std=c++23"), std::string::npos) << cmd; } } + +// THE PRECOMPILE HAS TO KNOW WHICH MACHINE, AND ONLY ONE OF TWO SOURCES EVER +// CARRIES IT. +// +// `stdModuleTargetFlags` reached only the CODEGEN command, on the reading that +// the first step needs headers and the second needs the machine. The first step +// needs both: a `--precompile` that does not say which target resolves the +// standard library's own `#include <__config>` against the BUILDING machine, +// and the error names a header rather than the missing flag. +// +// It was invisible while exactly two kinds of toolchain existed. A payload +// whose compiler IS its target needs no flag, and a PACKAGE-provided module +// carries the target inside `stdModuleFlags`. A payload whose compiler serves +// SEVERAL targets is a third kind and has neither -- one NDK clang++ compiles +// for both Android ABIs and is told which by `--target` alone. +TEST(ToolchainStdmod, ThePrecompileCarriesTheMachineWhenOnlyTargetFlagsHaveIt) { + auto tc = clang_toolchain(); + tc.targetTriple = "aarch64-linux-android"; + // What prepare_build sets for such a row: the machine, and nothing about + // include paths, because the SDK's own driver finds those. + tc.stdModuleTargetFlags = + " --target=aarch64-unknown-linux-android21 -D__BIONIC_CTYPE_INLINE="; + ASSERT_TRUE(tc.stdModuleFlags.empty()); + + auto cmds = clang::std_module_build_commands( + tc, "cache", "cache/pcm.cache/std.pcm", "", "-std=c++23"); + ASSERT_EQ(cmds.size(), 2u); + + // BOTH commands, not just the second. The precompile is the one that was + // missing it and the one whose failure names a header. + for (auto const& cmd : cmds) { + EXPECT_NE(cmd.find("--target=aarch64-unknown-linux-android21"), + std::string::npos) << cmd; + } + // And the bionic workaround reaches the step that parses the headers. + EXPECT_NE(cmds[0].find("-D__BIONIC_CTYPE_INLINE="), std::string::npos) + << cmds[0]; +} + +// AND IT IS NOT ADDED TWICE. `stdModuleFlags` is a SUPERSET of +// `stdModuleTargetFlags` when it is set at all -- its producer builds the +// machine part first and appends the include part -- so a precompile that +// concatenated both would put `-- on the command line twice. Taking +// the superset in preference is what keeps that from happening. +TEST(ToolchainStdmod, APackageProvidedModuleStillStatesTheMachineExactlyOnce) { + auto tc = clang_toolchain(); + tc.targetTriple = "aarch64-macos"; + tc.stdModuleTargetFlags = " -- + tc.stdModuleFlags = + " -- -nostdinc++ -isystem /pkg/include"; + + auto cmds = clang::std_module_build_commands( + tc, "cache", "cache/pcm.cache/std.pcm", "", "-std=c++23"); + ASSERT_EQ(cmds.size(), 2u); + + const auto count = [](std::string_view hay, std::string_view needle) { + std::size_t n = 0, at = 0; + while ((at = hay.find(needle, at)) != std::string_view::npos) { + ++n; at += needle.size(); + } + return n; + }; + EXPECT_EQ(count(cmds[0], "--s include path reaches the step that needs it, and only it. + EXPECT_NE(cmds[0].find("-isystem /pkg/include"), std::string::npos) + << cmds[0]; + EXPECT_EQ(cmds[1].find("-isystem /pkg/include"), std::string::npos) + << cmds[1]; +} diff --git a/tests/unit/test_toolchain_triple.cpp b/tests/unit/test_toolchain_triple.cpp index 3e609f611..c41578079 100644 --- a/tests/unit/test_toolchain_triple.cpp +++ b/tests/unit/test_toolchain_triple.cpp @@ -98,6 +98,40 @@ TEST(TripleRequest, TheOnlySupportedSiblingIsTaken) { EXPECT_FALSE(r.triple.envExplicit); } +TEST(TripleRequest, ABareLinuxTripleIsNeverCompletedToAndroid) { + // `aarch64-linux` asks for a C LIBRARY to be filled in. `android` shares + // the `arch-os` prefix because its kernel is Linux -- the modelling + // decision that makes every Linux-shaped answer in the tree right about it + // -- and it is not an alternative C library for the same platform. + // + // This became reachable when the Android rows stopped being `planned`: + // `aarch64-linux` then had two supported siblings and resolved as + // ambiguous, where it had completed to `aarch64-linux-musl` before. Both + // outcomes of that ambiguity are wrong -- refusing a request with an + // obvious answer, or answering it with bionic. + auto r = triple::resolve_request(*parse("aarch64-linux")); + EXPECT_EQ(r.triple.str(), "aarch64-linux-musl"); + EXPECT_TRUE(r.completedFromVocabulary); + EXPECT_FALSE(r.ambiguous); + // Not offered as a suggestion either: `siblings` is what the diagnostic + // prints, and naming it there would suggest building for another platform. + for (auto& s : r.siblings) + EXPECT_EQ(s.find("android"), std::string::npos) << s; + + // x86_64 has the same shape and a different supported set, so it exercises + // the exclusion independently rather than re-testing one row. + auto x = triple::resolve_request(*parse("x86_64-linux")); + for (auto& s : x.siblings) + EXPECT_EQ(s.find("android"), std::string::npos) << s; + + // AND THE WRITTEN SPELLING IS STILL HONOURED. The exclusion is about + // filling a gap, not about refusing a request -- an explicit env returns + // before the candidate loop runs. + auto explicitAndroid = triple::resolve_request(*parse("aarch64-linux-android")); + EXPECT_EQ(explicitAndroid.triple.str(), "aarch64-linux-android"); + EXPECT_FALSE(explicitAndroid.completedFromVocabulary); +} + TEST(TripleRequest, AWrittenSegmentIsARequestAndIsNotRevised) { // The escape hatch: writing the segment opts into the `planned` row, and the // tier gate then refuses something the user actually typed. @@ -515,3 +549,508 @@ TEST(Triple, EveryTableRowIsItsOwnCanonicalForm) { << info.canonical << " does not round-trip"; } } + +// ── The object-format axis, and the three platforms it exists for ─────────── +// +// The binary format used to be re-derived from `os` at every site that needed +// it, which is affordable only while the answer has two values. These hold the +// single derivation, because a site that misses a third value does not fail -- +// it silently answers ELF, which is what every `else` branch in the tree +// assumes. + +TEST(Triple, TheObjectFormatIsOneAnswerAndNotADerivation) { + EXPECT_EQ(parse("x86_64-linux-gnu")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("x86_64-linux-musl")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("aarch64-linux-android")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("riscv64-none-elf")->object_format(), ObjectFormat::Elf); + EXPECT_EQ(parse("x86_64-windows-gnu")->object_format(), ObjectFormat::Pe); + EXPECT_EQ(parse("x86_64-windows-msvc")->object_format(), ObjectFormat::Pe); + EXPECT_EQ(parse("aarch64-macos")->object_format(), ObjectFormat::MachO); + EXPECT_EQ(parse("aarch64-ios")->object_format(), ObjectFormat::MachO); + EXPECT_EQ(parse("wasm32-emscripten")->object_format(), ObjectFormat::Wasm); +} + +TEST(Triple, TheFormatQuestionIsNotTheOperatingSystemQuestion) { + // The two axes were conflated before `ObjectFormat` existed, and merging + // them is the mistake it replaces: a bare-metal image is ELF with no OS, + // and a wasm module has an OS-like layer and is not ELF. + auto bare = parse("riscv64-none-elf"); + EXPECT_TRUE(bare->is_freestanding()); + EXPECT_EQ(bare->object_format(), ObjectFormat::Elf); + + auto web = parse("wasm32-emscripten"); + EXPECT_FALSE(web->is_freestanding()); + EXPECT_TRUE(web->is_wasm()); +} + +TEST(Triple, IsPeAndIsMachOReadTheSingleAnswer) { + EXPECT_TRUE (parse("x86_64-windows-gnu")->is_pe()); + EXPECT_FALSE(parse("x86_64-windows-gnu")->is_mach_o()); + EXPECT_TRUE (parse("aarch64-ios")->is_mach_o()); + EXPECT_FALSE(parse("aarch64-ios")->is_pe()); + // The row that used to answer this by `os == "windows"` and would have + // answered ELF for wasm. + EXPECT_FALSE(parse("wasm32-emscripten")->is_pe()); + EXPECT_FALSE(parse("wasm32-emscripten")->is_mach_o()); +} + +TEST(Triple, AndroidIsAnEnvOnALinuxOs) { + auto t = parse("aarch64-linux-android"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->arch, "aarch64"); + // THE PLACEMENT IS THE MODELLING DECISION. The kernel is Linux, so ELF, + // the `unix` family and `nasm -f elf64` are all already right; an + // `os = "android"` would have made every one of them wrong by default. + EXPECT_EQ(t->os, "linux"); + EXPECT_EQ(t->env, "android"); + EXPECT_TRUE(t->is_android()); + EXPECT_EQ(t->family(), "unix"); + EXPECT_EQ(t->str(), "aarch64-linux-android"); + EXPECT_EQ(t->llvm_triple(), "aarch64-unknown-linux-android"); + + // `androideabi` is the 32-bit ARM spelling of the same env: the EABI half + // is the calling convention, which the arch segment already carries. + auto eabi = parse("armv7a-linux-androideabi"); + ASSERT_TRUE(eabi.has_value()); + EXPECT_EQ(eabi->env, "android"); + + // The env fill must not reach an Android request. `x86_64-linux` is still + // `gnu`; `x86_64-linux-android` is not. + EXPECT_EQ(parse("x86_64-linux")->env, "gnu"); + EXPECT_EQ(parse("x86_64-linux-android")->env, "android"); +} + +TEST(Triple, IosIsAppleWithoutBeingMacos) { + auto t = parse("aarch64-ios"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->os, "ios"); + EXPECT_TRUE(t->env.empty()); + EXPECT_EQ(t->family(), "unix"); + EXPECT_EQ(t->str(), "aarch64-ios"); + // Apple's own spelling of the architecture, as the macOS branch already + // produces. No deployment target is baked in: that flag belongs to the + // layer that owns the SDK, and a default here would be a second answer. + EXPECT_EQ(t->llvm_triple(), "arm64-apple-ios"); + + // A SITE THAT MEANS "APPLE" AND ASKS "macOS" GETS iOS WRONG IN THE + // DIRECTION THAT STILL LINKS, which is why the predicate exists. + EXPECT_TRUE(parse("aarch64-macos")->is_apple()); + EXPECT_TRUE(parse("aarch64-ios")->is_apple()); + EXPECT_FALSE(parse("aarch64-linux-musl")->is_apple()); + + // An effective triple carries the deployment target on this segment. + auto eff = parse("arm64-apple-ios17.0"); + ASSERT_TRUE(eff.has_value()); + EXPECT_EQ(eff->os, "ios"); +} + +TEST(Triple, EmscriptenIsAnOsSegmentAndNotAnEnv) { + auto t = parse("wasm32-emscripten"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->arch, "wasm32"); + // It names the platform layer a module is compiled against -- the POSIX + // emulation, the filesystem shim, the main loop -- which is the kind of + // thing `linux` names and not the kind of thing `musl` names. + EXPECT_EQ(t->os, "emscripten"); + EXPECT_TRUE(t->env.empty()); + EXPECT_EQ(t->str(), "wasm32-emscripten"); + EXPECT_EQ(t->llvm_triple(), "wasm32-unknown-emscripten"); + // `unix` on the test the predicate actually applies -- what API surface a + // source may assume -- rather than on a claim about wasm. + EXPECT_EQ(t->family(), "unix"); + // NASM is x86-family by construction and must decline rather than choose. + EXPECT_FALSE(t->nasm_format().has_value()); +} + +TEST(Triple, EachRowsTierMatchesTheEvidenceThatExistsForIt) { + // WHAT A TIER ASSERTS, AND THE THREE ANSWERS ARE DIFFERENT CLAIMS. + // + // verified an artefact was built AND RUN + // preview an artefact was built; nothing has executed it + // planned the vocabulary exists and nothing is wired + // + // A tier that moved on expectation would be the one thing this column + // cannot be, so each row below names the measurement behind it. + + { + // Measured 2026年09月11日 on linux-x86_64 with xim:emsdk 6.0.9: + // `mcpp run --target wasm32-emscripten` on a source that imports std + // printed `1-2-3`. Built and run, so `verified`. + auto t = parse("wasm32-emscripten"); + ASSERT_TRUE(t.has_value()); + auto* info = find_known_target(*t); + ASSERT_NE(info, nullptr); + EXPECT_EQ(info->tier, "verified"); + // A ROW THAT IS WIRED NAMES ITS PAYLOAD. Without the pin the row's + // tier was reachable only through an explicit + // `[target.wasm32-emscripten] toolchain = "..."` override, which is + // the escape hatch and not the support claim. + EXPECT_EQ(info->pin, "emsdk@6.0.9"); + // No `sysroot` column, and that is a statement: the SDK ships one, so + // there is no separate C library for the row to name. + EXPECT_TRUE(info->sysroot.empty()); + } + + // BOTH ANDROID ROWS ARE NOW WIRED, AND `preview` IS THE HONEST TIER. + // + // Measured 2026年09月11日 on linux-x86_64 with xim:android-ndk + // 30.0.16248370, from a source that imports std and no project + // vocabulary beyond `--target`: + // + // aarch64-linux-android -> ELF 64-bit LSB pie, ARM aarch64, + // interpreter /system/bin/linker64 + // x86_64-linux-android -> ELF 64-bit LSB pie, x86-64, same interpreter + // + // Neither has been EXECUTED, which is exactly the difference between this + // tier and the wasm row's: running one needs a device or an emulator. + // + // ONE PIN SERVES BOTH ROWS, which is the property the whole Android path + // rests on: the NDK names no arch, `--target` does, and that is why the + // std module's own precompile had to be told the target as well. + // ONE PIN, TWO TIERS, and the tiers differ by EXECUTION rather than by + // confidence in the build. `x86_64-linux-android` ran on the platform's + // own emulator (API 24 x86_64 image, KVM): `adb push` then + // `adb shell ./andtest` printed `1-2-3`, exit 0. The device row has no + // execution path from an x86_64 host -- Google's emulator refuses a + // foreign guest outright -- so it stays `preview`. + for (auto [name, tier] : {std::pair{"aarch64-linux-android", "preview"}, + std::pair{"x86_64-linux-android", "verified"}}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + EXPECT_EQ(t->str(), name); + auto* info = find_known_target(*t); + ASSERT_NE(info, nullptr) << name; + EXPECT_EQ(info->tier, tier) << name; + EXPECT_EQ(info->pin, "android-ndk@30.0.16248370") << name; + // Same statement the wasm row makes: the SDK ships the sysroot. + EXPECT_TRUE(info->sysroot.empty()) << name; + } + + // THE THREE APPLE ROWS STAY `planned`, AND THE BLOCKER IS NOT A PAYLOAD. + // The NDK is Apache-2.0 and Emscripten is MIT; the iPhoneOS and + // iPhoneSimulator SDKs ship inside Xcode and are neither. No amount of + // engine work moves these, which is why they carry no pin: there is + // nothing for a pin to name. + for (auto name : {"aarch64-ios", "aarch64-ios-sim", "x86_64-ios-sim"}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + EXPECT_EQ(t->str(), name); + auto* info = find_known_target(*t); + ASSERT_NE(info, nullptr) << name; + EXPECT_EQ(info->tier, "planned") << name; + EXPECT_TRUE(info->pin.empty()) << name; + EXPECT_TRUE(info->sysroot.empty()) << name; + } +} + +// A CAPABILITY PIN CANNOT BE OVERRIDDEN; A CONVENTION PIN CAN. +// +// Asserted exhaustively over the table rather than on examples, because the +// failure this guards against is a row JOINING the set without its refusal +// sentence being written. That has now happened twice -- wasm, then Android -- +// and each time the refusal explained a different row: "No gcc payload emits a +// PE with a musl C library", printed for a wasm target and then for an Android +// one, because a fourth case fell into an `else` written as the third's answer. +TEST(Triple, ExactlyTheseRowsHaveACapabilityPin) { + std::set capability; + for (auto& row : known_targets()) { + auto t = parse(row.canonical); + ASSERT_TRUE(t.has_value()) << row.canonical; + if (t->pin_is_capability()) capability.insert(std::string(row.canonical)); + } + // Every freestanding row, the PE+musl row, wasm, and both Android rows. + std::set expected{ + "aarch64-none-elf", "armv7a-none-eabi", "armv7a-none-eabihf", + "riscv32-none-elf", "riscv64-none-elf", "thumbv6m-none-eabi", + "thumbv7em-none-eabi", "thumbv7em-none-eabihf", "thumbv7m-none-eabi", + "thumbv8m.base-none-eabi", "thumbv8m.main-none-eabi", + "thumbv8m.main-none-eabihf", "x86_64-none-elf", + "x86_64-windows-musl", + "wasm32-emscripten", + "aarch64-linux-android", "x86_64-linux-android", + }; + EXPECT_EQ(capability, expected); + + // ANDROID IS THE ONE THAT DOES NOT FIT THE OTHERS' REASON, and that is why + // it was left out. The other entries are refused because the toolchain + // cannot emit the FORMAT; a stock clang emits aarch64 ELF perfectly well. + // What it cannot supply is bionic -- headers, per-API-level stubs, loader + // path -- and no package adds those to another compiler. + auto android = parse("aarch64-linux-android"); + ASSERT_TRUE(android.has_value()); + EXPECT_TRUE(android->pin_is_capability()); + EXPECT_FALSE(android->is_freestanding()); + EXPECT_FALSE(android->is_wasm()); + EXPECT_FALSE(android->is_pe() && android->is_musl()); + + // And a hosted row's pin stays a convention: an author who supplies the + // system may name any compiler. + auto musl = parse("x86_64-linux-musl"); + ASSERT_TRUE(musl.has_value()); + EXPECT_FALSE(musl->pin_is_capability()); +} + +// THE SIMULATOR IS A TARGET, NOT A RUNNER, AND IT NEEDS A SPELLING. +TEST(Triple, TheSimulatorRowsCarryEnvSimAndTheirOwnEffectiveTriple) { + // mcpp's three-field form. Rust spells this `aarch64-apple-ios-sim`; the + // difference is the vendor segment this table elides everywhere. + auto sim = parse("aarch64-ios-sim"); + ASSERT_TRUE(sim.has_value()); + EXPECT_EQ(sim->arch, "aarch64"); + EXPECT_EQ(sim->os, "ios"); + EXPECT_EQ(sim->env, "sim"); + EXPECT_EQ(sim->str(), "aarch64-ios-sim"); + EXPECT_TRUE(sim->is_apple()); + EXPECT_EQ(sim->object_format(), ObjectFormat::MachO); + EXPECT_EQ(sim->family(), "unix"); + + // APPLE'S OWN SPELLING PARSES TO THE SAME ROW. An effective triple clang + // prints carries `-simulator`, and a reader who pastes one must not be + // told mcpp has never heard of it. + auto apple = parse("arm64-apple-ios-simulator"); + ASSERT_TRUE(apple.has_value()); + EXPECT_EQ(apple->str(), "aarch64-ios-sim"); + + // TWO ROWS THAT MUST NOT SHARE AN IDENTITY. The device and the simulator + // have different SDKs and different objects; one identity would put two + // targets in one build directory. + auto device = parse("aarch64-ios"); + ASSERT_TRUE(device.has_value()); + EXPECT_NE(device->str(), sim->str()); + EXPECT_TRUE(device->env.empty()); + + // The effective triple differs too, and that is what the SDK selection + // downstream keys off. + EXPECT_EQ(sim->llvm_triple({}), "arm64-apple-ios-simulator"); + EXPECT_EQ(device->llvm_triple({}), "arm64-apple-ios"); + + // Both host arches, because the simulator runs the HOST's architecture: a + // single row would describe a simulator half the machines cannot run. + auto x86sim = parse("x86_64-ios-sim"); + ASSERT_TRUE(x86sim.has_value()); + EXPECT_EQ(x86sim->str(), "x86_64-ios-sim"); + EXPECT_EQ(x86sim->llvm_triple({}), "x86_64-apple-ios-simulator"); +} + +// THE API LEVEL IS FUSED ONTO THE ENV SEGMENT, AND IT IS NOT OPTIONAL. +TEST(Triple, AndroidFusesTheApiLevelAndBionicRequiresOne) { + auto t = parse("aarch64-linux-android"); + ASSERT_TRUE(t.has_value()); + // Canonical identity carries no level: one row serves every level, which + // is why the level is a project decision in `[target.
]` and not a + // multiplication of the table. + EXPECT_EQ(t->str(), "aarch64-linux-android"); + // The effective triple is where it lands. Measured: + // `clang -target aarch64-linux-android21 -print-effective-triple` + // answers `aarch64-unknown-linux-android21`. + EXPECT_EQ(t->llvm_triple("21"), "aarch64-unknown-linux-android21"); + EXPECT_EQ(t->llvm_triple("24"), "aarch64-unknown-linux-android24"); + // WITH NO LEVEL THE FORM IS STILL PRODUCED, and that is deliberate: this + // function composes, it does not decide. The refusal lives where the level + // is chosen, because bionic's own stops the build -- + // "Unversioned target triples are not supported!" -- and the default comes + // from the NDK's `meta/platforms.json` rather than from here. + EXPECT_EQ(t->llvm_triple({}), "aarch64-unknown-linux-android"); + // An effective triple with a level parses back to the canonical row. + auto back = parse("aarch64-unknown-linux-android21"); + ASSERT_TRUE(back.has_value()); + EXPECT_EQ(back->str(), "aarch64-linux-android"); +} + +// DOES THIS TARGET'S TOOLCHAIN ARRIVE WITH ITS OWN COMPLETE SYSTEM? +// +// The predicate exists because mcpp reconstructs a target's system by hand -- +// libc++'s headers, glibc's, the Linux UAPI headers, the C-runtime prefix, the +// loader -- and for a target whose SDK ships a sysroot every one of those is an +// answer competing with one the driver already has. Three independent sites +// read it, and each was found by the previous one's failure: +// +// host_compile_tokens this host's stdint.h reached a wasm compile +// resolve_link_model --dynamic-linker=...ld-linux-x86-64.so.2 reached wasm-ld +// discover_link_runtime_dirs the COMPILER's libatomic reached the ARTIFACT's link line +// +// Asserted as an exhaustive statement over the table rather than on examples, +// so a new row cannot join the set by accident or be left out of it. +TEST(Triple, OnlyTheSdkTargetsShipTheirOwnSysroot) { + std::set shipsOwn; + for (auto& row : known_targets()) { + auto t = parse(row.canonical); + ASSERT_TRUE(t.has_value()) << row.canonical; + if (t->has_own_sysroot()) shipsOwn.insert(std::string(row.canonical)); + } + EXPECT_EQ(shipsOwn, (std::set{ + "aarch64-linux-android", "x86_64-linux-android", "wasm32-emscripten"})); + + // `aarch64-ios` is NOT in the set, and that is the interesting exclusion. + // The iPhoneOS SDK does ship a sysroot -- but mcpp reaches it with + // `-isysroot`, which this predicate is not about: the question here is + // whether the DRIVER resolves the system without being told, and an + // ordinary clang pointed at an SDK does not. + auto ios = parse("aarch64-ios"); + ASSERT_TRUE(ios.has_value()); + EXPECT_FALSE(ios->has_own_sysroot()); +} + +// A CAPABILITY PIN CANNOT BE OVERRIDDEN, BECAUSE NOTHING ELSE CAN EMIT THE +// TARGET. A convention pin is a preference; this is a fact about the world. +// THE EFFECTIVE TRIPLE CARRIES THE PROJECT'S MINIMUM PLATFORM VERSION, AND THE +// CANONICAL ONE NEVER DOES. +// +// Two platforms fuse it and each names it in its own words: macOS's deployment +// target, Android's minimum API level. Measured on a real clang -- +// `-target aarch64-linux-android21 -print-effective-triple` answers +// `aarch64-unknown-linux-android21` -- so the level belongs on the ENV segment +// of the effective triple. +// +// Asserted as the PAIR, because the whole design is that the two differ: if +// `str()` ever carried the version, the output directory and `cfg()` would +// multiply per level and the table would need a row for each. +TEST(Triple, TheMinimumPlatformVersionReachesTheEffectiveTripleAndNotTheCanonicalOne) { + auto droid = parse("aarch64-linux-android"); + ASSERT_TRUE(droid.has_value()); + EXPECT_EQ(droid->str(), "aarch64-linux-android"); + EXPECT_EQ(droid->llvm_triple("24"), "aarch64-unknown-linux-android24"); + EXPECT_EQ(droid->llvm_triple("21"), "aarch64-unknown-linux-android21"); + // Unset is legal and means the NDK's own default -- what clang normalises + // when no level is given. + EXPECT_EQ(droid->llvm_triple(""), "aarch64-unknown-linux-android"); + // And the canonical form is unmoved by any of it. + EXPECT_EQ(droid->str(), "aarch64-linux-android"); + + // macOS, the platform this parameter already served, is unchanged. + auto mac = parse("aarch64-macos"); + ASSERT_TRUE(mac.has_value()); + EXPECT_EQ(mac->llvm_triple("15.2"), "arm64-apple-macos15.2"); + EXPECT_EQ(mac->str(), "aarch64-macos"); + + // AND NO OTHER ROW TAKES IT. One parameter serves both platforms, so the + // risk is a caller handing one platform's answer to another's row -- an + // ordinary Linux target must ignore it rather than fuse it. + auto lin = parse("x86_64-linux-gnu"); + ASSERT_TRUE(lin.has_value()); + EXPECT_EQ(lin->llvm_triple("24"), "x86_64-unknown-linux-gnu"); + auto musl = parse("aarch64-linux-musl"); + ASSERT_TRUE(musl.has_value()); + EXPECT_EQ(musl->llvm_triple("24"), "aarch64-unknown-linux-musl"); +} + +// THE FOUR-FIELD SPELLING IS WHAT EVERY OTHER TOOLCHAIN PRINTS, so refusing it +// is a cost with no design benefit. `em++ -v` passes +// `-target wasm32-unknown-emscripten`, rustc's table lists that spelling, and a +// user copying either into a manifest should be understood. +// +// mcpp's canonical form elides the vendor -- `unknown`, `pc` and `w64` carry no +// information for any row in the table -- so this is a normalisation and not a +// second vocabulary: `str()` returns the three-field form either way, which is +// what keeps the output directory, `cfg()` and the ABI tag single-valued. +TEST(Triple, TheFourFieldSpellingParsesToTheSameCanonicalTriple) { + struct Case { const char* spelled; const char* canonical; }; + for (auto [spelled, canonical] : { + Case{"wasm32-unknown-emscripten", "wasm32-emscripten"}, + Case{"aarch64-apple-ios", "aarch64-ios"}, + Case{"aarch64-unknown-linux-android", "aarch64-linux-android"}, + Case{"x86_64-unknown-linux-gnu", "x86_64-linux-gnu"}, + Case{"x86_64-pc-windows-msvc", "x86_64-windows-msvc"}, + Case{"aarch64-unknown-linux-musl","aarch64-linux-musl"}, + }) { + auto t = parse(spelled); + ASSERT_TRUE(t.has_value()) << spelled; + EXPECT_EQ(t->str(), canonical) << spelled; + // And the three-field form still parses to itself, so accepting the + // longer spelling did not make the canonical one a second dialect. + auto c = parse(canonical); + ASSERT_TRUE(c.has_value()) << canonical; + EXPECT_EQ(c->str(), canonical) << canonical; + } +} + +TEST(Triple, WasmJoinsTheCapabilityPinsBecauseNothingElseEmitsIt) { + auto wasm = parse("wasm32-emscripten"); + ASSERT_TRUE(wasm.has_value()); + EXPECT_TRUE(wasm->pin_is_capability()) + << "a declared gcc@16.1.0 would otherwise override emsdk@6.0.9 and " + "fail inside a compiler that cannot emit WebAssembly"; + + // The two that were there before, unchanged. + EXPECT_TRUE(parse("riscv64-none-elf")->pin_is_capability()); + EXPECT_TRUE(parse("x86_64-windows-musl")->pin_is_capability()); + // And an ordinary hosted row is still a convention: a project may name + // whichever compiler it likes for its own Linux. + EXPECT_FALSE(parse("x86_64-linux-musl")->pin_is_capability()); + // ANDROID IS ONE NOW, AND THIS ASSERTION PREDICTED ITS OWN EXPIRY. It + // read EXPECT_FALSE, with the note "the row carries no pin yet [...] it + // moves when the row does" -- the row moved, so it did. + // + // The reason is not the other three's. They are refused because the + // toolchain cannot emit the FORMAT; a stock clang emits aarch64 ELF + // perfectly well. What it cannot supply is bionic, which lives inside the + // NDK and is not packaged onto another compiler. + EXPECT_TRUE(parse("aarch64-linux-android")->pin_is_capability()); + EXPECT_TRUE(parse("x86_64-linux-android")->pin_is_capability()); + // The iOS rows are NOT capability pins, and the distinction is worth an + // assertion: they carry no pin at all, so there is nothing to override and + // nothing to refuse. Their tier is what stops a build, not their pin. + EXPECT_FALSE(parse("aarch64-ios")->pin_is_capability()); + EXPECT_FALSE(parse("aarch64-ios-sim")->pin_is_capability()); +} + +TEST(Triple, TheCanonicalSpellingIsNotSEARCHABLEForAVENDORNAME) { + // WHY A SUBSTRING TEST ON THE CANONICAL TRIPLE IS WRONG, stated as a fact + // about the vocabulary rather than as a comment somewhere else. + // + // Two sites derived the object format by looking for "apple" / "darwin" / + // "windows" / "mingw" in `plan.toolchain.targetTriple`. That string is + // mcpp's CANONICAL spelling, and `aarch64-macos` contains none of those + // words -- so an explicit `--target aarch64-macos`, a `verified` row, was + // recorded and linked as ELF. A NATIVE macOS build was right by a + // different branch (an empty triple), which is why the two paths through + // one function disagreed and only the exercised one was correct. + // + // The words appear in the LLVM spelling, which is a different string and + // the reason the mistake is easy to make: + // + // aarch64-macos -> arm64-apple-macos14.0 + // ^ the identity ^ what clang is given + for (auto name : {"aarch64-macos", "x86_64-macos", "aarch64-ios"}) { + auto t = parse(name); + ASSERT_TRUE(t.has_value()) << name; + const std::string canonical = t->str(); + EXPECT_EQ(canonical.find("apple"), std::string::npos) << canonical; + EXPECT_EQ(canonical.find("darwin"), std::string::npos) << canonical; + // And the format is right anyway, because it is asked of the fields. + EXPECT_EQ(t->object_format(), ObjectFormat::MachO) << canonical; + // The LLVM spelling is where the vendor name lives. + EXPECT_NE(t->llvm_triple().find("apple"), std::string::npos) + << t->llvm_triple(); + } + // The one family the substring test got right, and only by luck: the + // canonical spelling happens to carry the OS name. + EXPECT_NE(std::string(parse("x86_64-windows-gnu")->str()).find("windows"), + std::string::npos); +} + +TEST(Triple, EveryKnownRowHasAnObjectFormatAndNoneFallsThrough) { + // THE DENOMINATOR IS THE TABLE. A row added without an answer here would + // otherwise be covered by a test whose name says every row is -- and the + // answer it would get is ELF, because ELF is what every `else` branch in + // the tree assumes. + std::size_t elf = 0, macho = 0, pe = 0, wasm = 0; + for (auto const& row : known_targets()) { + auto t = parse(row.canonical); + ASSERT_TRUE(t.has_value()) << row.canonical; + EXPECT_EQ(t->str(), row.canonical) << "a row that is not its own canonical form"; + switch (t->object_format()) { + case ObjectFormat::Elf: ++elf; break; + case ObjectFormat::MachO: ++macho; break; + case ObjectFormat::Pe: ++pe; break; + case ObjectFormat::Wasm: ++wasm; break; + } + } + // Each format has at least one row, which is what makes the axis worth + // having: a fourth value with no row would be an enum nothing produces. + EXPECT_GT(elf, 0u); + EXPECT_GT(macho, 0u); + EXPECT_GT(pe, 0u); + EXPECT_EQ(wasm, 1u) << "wasm32-emscripten is the only wasm row today"; + EXPECT_EQ(elf + macho + pe + wasm, known_targets().size()); +}

AltStyle によって変換されたページ (->オリジナル) /