unused std.Io functions are not optimized out leading to executable bloat #31421

Zig Version

0.16.0-dev.2682+02142a54d

Steps to Reproduce and Observed Behavior

I am using the Linux x86_64 versions of the compilers listed at https://ziglang.org/download/.

The file that I am compiling is simply this:

// main.zig
pub fn main() void {}

This is how I compile the file in each version:

./zig-x86_64-linux-0.15.2/zig build-exe -O ReleaseSmall -fstrip --name oldmain main.zig
./zig-x86_64-linux-0.16.0-dev.2682+02142a54d/zig build-exe -O ReleaseSmall -fstrip --name newmain main.zig

There is a 21 times increase in the outputted executable size:

4.9K	oldmain
101K	newmain

Looking at the executable dumps this size increase seems to be related to the new changes in std.Io and std.process but I may be wrong about that.

Expected Behavior

The executable size produced by 0.16.0 should be around that of 0.15.2.

+1

pubfnmain()void{}

zig build-exe main.zig -OReleaseSmall -fstrip -target x86_64-linux -fsingle-threaded
wc -c main
113256 main

zig-0.15.2 build-exe main.zig -OReleaseSmall -fstrip -target x86_64-linux -fsingle-threaded
wc -c main
1912 main

On master:

pubconstpanic=@import("std").debug.no_panic;pubfnmain()void{}

pubconstpanic=@import("std").debug.simple_panic;pubfnmain()void{}

$ zig build-exe -OReleaseSmall no_panic.zig
$ zig build-exe -OReleaseSmall simple_panic.zig
$ ls -sh no_panic simple_panic
8.0K no_panic 132K simple_panic

--- a/lib/std/Thread.zig
+++ b/lib/std/Thread.zig
@@ -1741,6 +1741,6 @@ pub fn maybeAttachSignalStack() void {
 }, null) catch |err| switch (err) {
 error.SizeTooSmall => unreachable, // `std.options.signal_stack_size` must be sufficient for the target
 error.PermissionDenied => unreachable, // called `maybeAttachSignalStack` from a signal handler
- error.Unexpected => @panic("unexpected error attaching signal stack"),
+ error.Unexpected => unreachable,
 };
 }

$ ls -sh before after
8.0K after 132K before

Seems to be fixed.

128.8 KB 28 Mar newmain_0.16.0-dev.2960
 5.6 KB 28 Mar newmain_0.16.0-dev.3006
 4.8 KB 28 Mar oldmain

above fix included in 9ac1386c10/std/Thread.zig

Oops, did not mean to commit that, but there's also no reason to revert it.

should it be closed? still 2.5x increase

zig build-exe main.zig -OReleaseSmall -fstrip -target x86_64-linux -fsingle-threaded
wc -c main
5080 main

zig-0.15.2 build-exe main.zig -OReleaseSmall -fstrip -target x86_64-linux -fsingle-threaded
wc -c main
1912 main

The main thing this was tracking was the dead code bloat from unused Io functions.

The remaining code delta can be fixed with:

--- a/lib/std/Io/Threaded.zig
+++ b/lib/std/Io/Threaded.zig
@@ -57,14 +57,9 @@ use_fcopyfile: UseFcopyfile = .default,
 use_fchmodat2: UseFchmodat2 = .default,
 disable_memory_mapping: bool,
 
-stderr_writer: File.Writer = .{
- .io = undefined,
- .interface = File.Writer.initInterface(&.{}),
- .file = if (is_windows) undefined else .stderr(),
- .mode = .streaming,
-},
-stderr_mode: Io.Terminal.Mode = .no_color,
 stderr_writer_initialized: bool = false,
+stderr_writer: File.Writer = undefined,
+stderr_mode: Io.Terminal.Mode = .no_color,
 stderr_mutex: Io.Mutex = .init,
 stderr_mutex_locker: std.Thread.Id = Thread.invalid_id,
 stderr_mutex_lock_count: usize = 0,
@@ -13804,8 +13799,12 @@ fn tryLockStderr(userdata: ?*anyopaque, terminal_mode: ?Io.Terminal.Mode) Io.Can
 fn initLockedStderr(t: *Threaded, terminal_mode: ?Io.Terminal.Mode) Io.Cancelable!Io.LockedStderr {
 if (!t.stderr_writer_initialized) {
 const io_t = io(t);
- if (is_windows) t.stderr_writer.file = .stderr();
- t.stderr_writer.io = io_t;
+ t.stderr_writer = .{
+ .io = io_t,
+ .interface = File.Writer.initInterface(&.{}),
+ .file = .stderr(),
+ .mode = .streaming,
+ };
 t.stderr_writer_initialized = true;
 t.scanEnviron();
 const NO_COLOR = t.environ.exist.NO_COLOR;

And the remaining data delta is a single global variable, which fails to be eliminated because of the argv0 and environ stores.

"empty main" is a good benchmark; however, let's see what the situation is with a basic "hello-world" sample.

hello.zig:

conststd=@import("std");pubfnmain()void{std.debug.print("Hello, world!",.{});}

Build command:

zigbuild-exehello.zig-OReleaseSmall-targetx86_64-windows-gnu-fstrip-fsingle-threaded--name(zigversion|strtrim)

And results:

╭───┬───────────────────────────────┬──────┬──────────┬───────────────╮
│ # │ name │ type │ size │ modified │
├───┼───────────────────────────────┼──────┼──────────┼───────────────┤
│ 0 │ 0.12.0.exe │ file │ 3,0 kB │ 2 minutes ago │
│ 1 │ 0.14.0.exe │ file │ 4,0 kB │ 2 minutes ago │
│ 2 │ 0.15.0-dev.451+a843be44a.exe │ file │ 4,0 kB │ 2 minutes ago │
│ 3 │ 0.15.1.exe │ file │ 11,7 kB │ a minute ago │
│ 4 │ 0.16.0-dev.1484+d0ba6642b.exe │ file │ 14,8 kB │ a minute ago │
│ 5 │ 0.16.0-dev.3128+ad7a02822.exe │ file │ 365,0 kB │ a minute ago │
╰───┴───────────────────────────────┴──────┴──────────┴───────────────╯

As you can see, the binary size on the nightly is over 100 times larger than on 0.12.0.

This will be a known regression in the 0.16.x release, with plan to address it in the future.

For the hello world from the langref:

conststd=@import("std");pubfnmain(init:std.process.Init)!void{trystd.Io.File.stdout().writeStreamingAll(init.io,"Hello, World!\n");}

the only things currently causing unused std.Io functions to not be optimized out are:

--- a/lib/std/Io/File.zig
+++ b/lib/std/Io/File.zig
@@ -610,7 +610,7 @@ pub fn writerStreaming(file: File, io: Io, buffer: []u8) Writer {
 
 /// This is a low-level API that calls the `Io` interface function directly.
 /// For a higher level API, see `writerStreaming`.
-pub fn writeStreaming(file: File, io: Io, header: []const u8, data: []const []const u8, splat: usize) Writer.Error!usize {
+pub inline fn writeStreaming(file: File, io: Io, header: []const u8, data: []const []const u8, splat: usize) Writer.Error!usize {
 return (try io.operate(.{ .file_write_streaming = .{
 .file = file,
 .header = header,
diff --git a/lib/std/Io/Threaded.zig b/lib/std/Io/Threaded.zig
index 0bec8a068f..460fa6470e 100644
--- a/lib/std/Io/Threaded.zig
+++ b/lib/std/Io/Threaded.zig
@@ -13809,6 +13809,7 @@ fn tryLockStderr(userdata: ?*anyopaque, terminal_mode: ?Io.Terminal.Mode) Io.Can
 }
 
 fn initLockedStderr(t: *Threaded, terminal_mode: ?Io.Terminal.Mode) Io.Cancelable!Io.LockedStderr {
+ if (true) @trap();
 if (!t.stderr_writer_initialized) {
 const io_t = io(t);
 if (is_windows) t.stderr_writer.file = .stderr();

Unused operations are also not optimized out, but that's a different problem.

@jacobly This means that when writeStreaming is inlined, the compiler sees the concrete operate call, can statically resolve the VTable dispatch and eliminate everything that is not the concrete file_write_streaming path?