Define a distinct Zig libc ABI #35379

Supersedes/migrated from: https://github.com/ziglang/zig/issues/24560

When filling in Zig code (#30978) for established libc ABIs such as musl, MinGW-w64, etc, we cannot deviate from those ABIs in any observable way, which means we inherit any quirks that they have, and we cannot extend the API surface in any way. We also would like to define a freestanding subset of Zig libc, but this can't be done under the names of those established libcs either.

So the task here is to define a completely distinct Zig libc ABI - think aarch64-linux-zig, x86_64-freestanding-zig, and so on. (In https://github.com/ziglang/zig/issues/20690 terms, the zig bit would be the api component.) With this ABI, we have the freedom to:

• Break with any legacy nonsense in existing libcs.
• Define consistent C ABI rules across targets.
• Offer a freestanding subset.
• Potentially extend the API surface.
• Author our own libc headers.

I'm currently cooking up an initial proposal for the ABI, which should be ready soon.

Would this also define consistent calling convention?

@Cloudef wrote in #35379 (comment):

Would this also define consistent calling convention?

That depends on what you mean by "calling convention".

For example x86_64-linux-zig and x86_64-windows-zig would call the external zig ABI functions the same way. For -freestanding it is often what compiler decides to do (usually the same as linux).

Can we get away with changing the size and format of long double? Sure.

Can we get away with changing the size of long? No; the Windows SDK (and thus ntdll.dll/kernel32.dll) has a hard requirement that it's always equal to int.

So again, it depends on what you mean. But I can say that, in general, we have no ability to change how types get passed in registers and the stack; we can only change their basic properties like size and alignment, and even then, the platform may impose hard requirements on us, as is the case on Windows.

I was mainly wondering how far this zig abi definition goes. So there is still the arch-os specific baggage, and -zig would only define the libc bits?

I personally would be interested in very clearly documented calling convention. As my use case is loading cross-OS binaries, and my only way to cope is to make sure everyone does whatever LLVM happens to do and I don't pass or return too complex types :). I presume this is out of scope here.

@Cloudef wrote in #35379 (comment):

I was mainly wondering how far this zig abi definition goes. So there is still the arch-os specific baggage, and -zig would only define the libc bits?

What we can do is things like making char signedness consistent, having a consistent size for long double, require IEEE floating point, and a few other things like that. However, we can't do anything about ABI requirements that Microsoft have decreed for the Windows SDK, even if we wanted to. (And we would very much like to fix long on Windows, but alas.)

@Cloudef wrote in #35379 (comment):

I personally would be interested in very clearly documented calling convention. As my use case is loading cross-OS binaries, and my only way to cope is to make sure everyone does whatever LLVM happens to do and I don't pass or return too complex types :). I presume this is out of scope here.

I think it would be problematic to pursue custom calling convention rules -- in the llvm/lib/Target/*/*CallingConv.td and clang/lib/CodeGen/Targets/*.cpp sense -- for the Zig libc ABI because it would require non-trivial compiler work on all targets - work that GCC and Clang may not even be interested in accepting upstream. On the other hand, adjusting things like type sizes and alignments is a fairly simple change that should be uncontroversial because most environments define some custom rules in this area.

Ultimately, there's not much point in linking libc in the first place if you can't also link C code into your program. So we have to approach this task with a healthy dose of pragmatism so that we can get the major C compilers on board.

Besides that, the Zig libc ABI spec would explode in size if we had to fully document calling conventions for every target.

Yeah this is all reasonable. The scope is pretty clear to me now.

the freestanding subset here in particular would remove one of my last remaining external dependencies and be very useful in general for osdev, where linking to small freestanding c libraries while using only the freestanding header subset and not fully linking to libc is a very common pattern (example libraries being uACPI or flanterm)

Consistent C ABI rules sounds great.

If I understood correctly, this would make arm32-freestanding-zig an attractive choice for projects where you deeply care about reproducible behavior across targets (like in the safety critical embedded systems industry).

In a way, this makes me think of WG14 C standard board member Eskil Steenberg's dependable c project. It defines a subset of rules to make C "dependable". I wonder how much of those issues could be addressed by the Zig libc ABI.