Moves these subcommands to the Maker process:
- zig build
- zig fetch
- zig init
- zig libc
This means that large parts of what used to be included in the compiler executable are now shipped in source form instead, including:
- package fetching logic
- http client & networking
- TLS (Transport Layer Security) and associated crypto
- git protocol
- xz, gzip, zstd, flate, zip
- parsing, validation, and otherwise dealing with build.zig.zon files
It also means you can now patch that functionality without rebuilding the compiler, making it easier for users and contributors to tinker.
Furthermore, it means that package management in zig now has safety checks enabled when doing networking, since Maker executable is compiled with ReleaseSafe.
The original motivation for doing this was in relation to exposing a build server in order to unblock ZLS after #35428 broke it. Before, the Maker process would have had to exit and be restarted whenever changes were detected in configurer source files, but now it can be long lived even when build.zig files receive source updates.
This changeset also puts the main maker loop inside another, outer loop that gets rerun when configuration changes (or when configurer source files change). Therefore this is progress towards #35460, #35473, and #35500 but it does not actually complete them.
Data Points
- Zig executable binary size: shrinks 4% from 14.1 to 13.5 MiB (no llvm, ReleaseSmall)
Downsides of This Approach:
- default
--seedmust always be computed byzig buildeven when user overrides it later --zig-lib-dirand similar args cannot be implemented; that configuration would only be supported to be set via env var- no longer doing first-time compilation of maker and compilation/execution of configurer simultaneously
- no longer checking for cache hit of configuration and cache hit of maker simultaneously
-jargument no longer affects compilation of maker (https://github.com/ziglang/zig/issues/20274)- no longer supporting
--maker-opt(useZIG_DEBUG_CMDinstead)
Followup Work
- finish implementing
confPathDepToCachePath - handle
directoryandmetadatamode of configuration.path_deps insideconfigure - use a response file for the configure argv if it is too long
- "TODO use multireader instead"
- add the file system inputs from fetching to configuration cache manifest (and watching)
- build.zig.zon files
- forks
- handle failed but with cache intact cases
- TODO serve a different message when the configuration changes
- TODO: handle DependencyLoopDetected as error.FailedButCacheIntact and handle InsufficientMemory as error.AlreadyReported
- fix warning(fetch): failed deleting temporary directory /home/andy/.cache/zig/tmp/.tmp-06068ad36066a5e1: DirNotEmpty (affects master branch too)
- also migrate
zig stdinto maker process - #35460
- #35473
- #35500
- investigate why panic in maker process prints "stack tracing disabled"
Moves these subcommands to the Maker process:
* zig build
* zig fetch
* zig init
* zig libc
This means that large parts of what used to be included in the compiler executable are now shipped in source form instead, including:
* package fetching logic
* http client & networking
* TLS (Transport Layer Security) and associated crypto
* git protocol
* xz, gzip, zstd, flate, zip
* parsing, validation, and otherwise dealing with build.zig.zon files
It also means you can now patch that functionality without rebuilding the compiler, making it easier for users and contributors to tinker.
Furthermore, it means that package management in zig now has safety checks enabled when doing networking, since Maker executable is compiled with ReleaseSafe.
The original motivation for doing this was in relation to exposing a build server in order to unblock ZLS after #35428 broke it. Before, the Maker process would have had to exit and be restarted whenever changes were detected in configurer source files, but now it can be long lived even when build.zig files receive source updates.
This changeset also puts the main maker loop inside another, outer loop that gets rerun when configuration changes (or when configurer source files change). Therefore this is progress towards #35460, #35473, and #35500 but it does not actually complete them.
## Data Points
* Zig executable binary size: shrinks 4% from 14.1 to 13.5 MiB (no llvm, ReleaseSmall)
## Downsides of This Approach:
- default `--seed` must always be computed by `zig build` even when user overrides it later
- `--zig-lib-dir` and similar args cannot be implemented; that configuration would only be supported to be set via env var
- no longer doing first-time compilation of maker and compilation/execution of configurer simultaneously
- no longer checking for cache hit of configuration and cache hit of maker simultaneously
- `-j` argument no longer affects compilation of maker (https://github.com/ziglang/zig/issues/20274)
- no longer supporting `--maker-opt` (use `ZIG_DEBUG_CMD` instead)
## Followup Work
- finish implementing `confPathDepToCachePath`
- handle `directory` and `metadata` mode of configuration.path_deps inside `configure`
- use a response file for the configure argv if it is too long
- "TODO use multireader instead"
- add the file system inputs from fetching to configuration cache manifest (and watching)
- build.zig.zon files
- forks
- handle failed but with cache intact cases
- TODO serve a different message when the configuration changes
- TODO: handle DependencyLoopDetected as error.FailedButCacheIntact and handle InsufficientMemory as error.AlreadyReported
- fix warning(fetch): failed deleting temporary directory /home/andy/.cache/zig/tmp/.tmp-06068ad36066a5e1: DirNotEmpty (affects master branch too)
- also migrate `zig std` into maker process
- #35460
- #35473
- #35500
- investigate why panic in maker process prints "stack tracing disabled"