ziglang/zig
262
6.0k
Fork
You've already forked zig
777

Windows: Prefer the Native API over Win32 #31131

Closed
opened 2026年02月05日 21:06:57 +01:00 by The-King-of-Toasters · 9 comments

As Zig has evolved, it has become a target to avoid calling Win32 APIs from kernel32.dll etc., instead using lower-level ones in ntdll.dll. This has been discussed in #1840, but this issue serves as its replacement, and a reformulation of why this practice is preferred.

Firstly: I am not a core Zig developer, and I do not speak for the team. I have made some pull requests in this area, and have helped out core developers to do the same. I am also not an expert on Windows InternalsTM, but I do read what other experts write and use their tools.


On modern Windows, the Win32 APIs are implemented as a "Subsystem"1 of the operating system, with its implementation relying on the "Native APIs" provided by the kernel. User-mode access to the Native API is done by calling functions exported by ntdll, which issues system calls to the NT kernel with the same arguments.

Comparing the comprehensive Win32 API reference against the incidentally documented Native APIs, its clear which one Microsoft would prefer you use. The native API is treated as an implementation detail, whilst core parts of Windows' backwards compatibility strategy are implemented in Windows subsystem2 .

Why then would anyone use the Native API? The best answer I've found is in Windows Native API Programming3 :

Using the native APIs can have the following benefits:

  • Performance - using the native API bypasses the standard Windows API, thus removing a software layer, speeding things up.
  • Power - some capabilities are not provided by the standard Windows API, but are available with the native API.
  • Dependencies - using the native API removes dependencies on subsystem DLLs, creating potentially smaller, leaner executables.
  • Flexibility - in the early stages of Windows boot, native applications (those dependent on NtDll.dll only) can execute, while others cannot.

That said, there are potential disadvantages to using the native API:

  • It’s mostly undocumented, making it more difficult to use from the get go. This is where this book comes in!
  • Being undocumented also means that Microsoft can make changes to the native API without warning, and no one can complain. That said, removal of capabilities from the native API or modification of existing functionality are rare. This is because some of Microsoft’s own tools and applications leverage the native API.

The points on "performance" and "power" are more relevant for Zig for a couple reasons:

  • The backwards compatibility features come at the cost of unexpected (to the programmer) DLL loading, which in turn requires heap allocation, file loading and use of critical sections. These are things that Zig code should handle by itself, but a practical benefit is a reduction of memory/file errors that occur when the system is overloaded. This has been observed on the Windows CI machines.
  • Most Win32 APIs return a BOOL for success/failure, with the failure code needing to be fetched by calling GetLastError. Native APIs return an NTSTATUS directly, which is a richer set of error/informational codes. Using the native APIs avoids having to call into kernel324 and allows for code to handle errors in the same switch where a function is called.
  • Speaking of types, the ones used for Native API programming are just better than the Win32 ones:
    • Times/timeouts are specified as LARGE_INTEGERs, a signed 64-bit time value with a 100ns resolution. An upgrade compared to the dwMilliseconds found in some Win32 functions, or FILETIMEs which require bit-shifting.
    • ACCESS_MASKs, used when setting/querying rights on a kernel object, map nicely to Zig's packed structs.
  • Many Native APIs take an IO_STATUS_BLOCK (essentially one part of the Win32 LPOVERLAPPED), but these are often hidden by the Win32 APIs.
  • More features are exposed by the Native APIs than the Win32 APIs that call them. In addition, there are some replacement/extended APIs that are unused or have no equivalent:
    • ReadDirectoryChangesW is implemented using NtNotifyChangeDirectoryFile. On my Windows 11 machine, this is a thin wrapper around NtNotifyChangeDirectoryFileEx, which exposes more information via FILE_NOTIFY_FULL_INFORMATION.
    • NtFlushBuffersFileEx allows for more granular flushing than FlushBuffersFile, and is used by e.g. PostgreSQL to provide the equivalent of fdatasync.

Does this mean that the Zig standard library will replace every Win32 API with the corresponding Native ones? No, and don't expect it ever will. There are certain parts of the Win32 API that are too complicated and/or undocumented enough that it's not worth the effort. These include:

  • (削除) The console APIs, where the implementation was completely rewritten in Windows 8+ and remains under-researched. (削除ここまで) Nevermind, this happened as I was writing this.
  • Bypassing the Winsock API and using \Device\Afd directly.
  • Anything to do with loading libraries.
  • Creating Win32 processes with NtCreateUserProcess. Heed this note from Windows Native API Programming:

There have been several attempts in the Infosec community to utilize RtlCreateUserProcess and/or NtCreateUserProcess to allow running Windows subsystem applications, with varying degrees of success. Part of the problem is the need to communicate with the Windows Subsystem process (csrss.exe) to notify it of the new process and thread. This turns out to be fragile, as different Windows versions may have somewhat different expectations.

Everything else is fair game:

  • Some Win32 functions are ABI compatible with the Native ones, thus the implementation is a set of jumps from kernel32.dll -> kernelbase.dll -> ntdll.dll -> ntoskrnl.exe. #25766 eliminated all of the remaining "forwarders", and you can see how e.g. ReleaseSRWLockExclusive forwards to RtlReleaseSRWLockExclusive.
  • Others are "thin wrappers" that call the native APIs and convert types/errors:
    • CopySid is just RtlCopySid, NT_SUCCESS to check the error code and BaseSetLastNTError for error mapping.
    • Some convert types like the Win32 BOOL (i32) into the Native BOOLEAN (i8) before calling the Native APIs. See SetSystemTimeAdjustment, AddAuditAccessAce for examples.
  • Some are "combination" wrappers that call multiple Native APIs that can be split into multiple parts:
    • CreateIoCompletionPort calls NtCreateIoCompletion if ExistingCompletionPort is NULL, then associates the FileHandle with the port and CompletionKey via NtSetInformationFile.
    • SetHandleInformation queries the status of hObject via NtQueryObject before calling NtSetInformationObject.
  • Finally, some functions setup the aforementioned compatibility features before calling Native APIs:
    • ReadDirectoryChangesW may create an activation context to load a Side-by-side assembly if lpCompletionRoutine is not NULL.
    • GetOverlappedResult always calls SbSelectProcedure if bWait is TRUE. This is part of the SwitchBack system introduced in Windows Vista.

You, dear reader, may have some doubts as to whether this is all worth it. As someone who has done this work before, let me answer some common questions:

I use the functions in std.os.windows.kernel32 directly, what will happen if I upgrade Zig and they are removed?

If a re-implementation wasn't made, then a compile error most likely. The standard library should contain only the host functions/types needed to support itself. We recommend copying the necessary function definitions/types into your project, or using something like zigwin32.

Doing this breaks compatibility with older versions of Windows, which is important for me.

True! But that is not (currently) a goal for Zig; standard library support is based on the earliest platform release supported by its developers. For Windows that means 10 and 11; and the server variants based on them. If you value backwards compatibility, replace calls to the standard library with Win32 ones instead. You can even replace your main function with wWinMain!. This practice is encouraged, as most of the standard library should work on free-standing environments.

Won't this get flagged by anti-virus scanners as suspicious?

Unfortunately, yes. We consider this a problem for the anti-virus scanners to solve.

Microsoft are free to change the Native API at will, and you will be left holding both pieces when things break.

While this can happen, we have not (yet) been affected by any changes in the Win32 -> Native layers. Also, both API sets are quite stable and functionality is rarely removed. If anything we prefer using more modern Native APIs than older ones. That said, if there are issues then we prefer to handle it in the standard library. Calling the Native APIs can bypass Windows-on-Windows support, but we have not seen significant issues by doing so.

I want to run my Zig programs on Wine. What if Wine's implementation differs from Windows, or is stubbed out?

We consider this a bug in Wine, and not something to work around unless it significantly impacts us - e.g. CI failures on Linux due to running the test suite with Wine. The cases I know of involve the behaviour of NtQueryObject and the console API re: using the UTF-8 codepage. The former requires a small workaround, the latter led to the standard library continuing to rely on kernel32.

Doing this requires observing the same implementation details as Windows/Wine/ReactOS. This is a fool's errand for a programming language.

True! But that is our cross to bear. The standard library most definitely contains different logic re: path-name validation than Windows. This means that WoW64 path redirection most likely doesn't work when compiling for x86-windows. On the other hand, this allows the standard library to handle issues arising from BadBatBut.

Using undocumented APIs like this will just lead to runtime errors, SEH exceptions, and other nasal daemons.

If some API is more sophisticated than a forwarder/wrapper, we investigate if the replacement is worth pursuing in the first case. We prefer the Native API, not require it; if you've encountered an issue due to this policy: report it! We are mostly reasonable people. But arguments like "Microsoft doesn't want you to do this" or "obviously this will lead to problems later on!" don't count.

Doing this means that Microsoft can't change/improve their internal APIs!

While well-intentioned, this is incorrect for a couple reasons:

  • The design of the Native APIs are built to allow for future functionality. Instead of tightly coupling a structure to a system call as you often see for the *Nixen, they are passed in as a tuple of {OperationEnum, struct pointer, length of struct}. For example, NtQueryInformationFile takes a FILE_INFORMATION_CLASS enum, and there has been numerous additions made over the many releases of Windows. The kernel also validates the structure length and will return an error if it is incorrect/too small. This can be used to allocate enough storage for flexible structures if the API takes a return length pointer.
  • If a new API is needed, the old one is never removed, instead:
    • The new API is exported via ntdll, and a superset of the old one with the addition of one-or-more parameters. You can distinguish them by their Ex suffix (or Ex2 if the first time wasn't enough). The same thing happens in Win32 land where you have three MapViewOfFiles (four if you count the Numa variant).
    • The Win32 code is updated to call the new API, or the old API calls the new one. See the comment about ReadDirectoryChangesW.
    • Alternatively, the old code isn't changed at all and multiple Native APIs will live... side-by-side.
  • The existence of the Wine and ReactOS projects, along with Microsoft's own Side-by-side/Switchback systems, proves that has this already happened. There are large swathes of programs/libraries that depend on this functionality, and it is in everybody's best interest that they are kept available.

  1. Older versions of Windows NT had additional subsystems to support running POSIX and OS/2 applications, but these have since been removed. Aside from the introduction of WSL 1, the separation between Win32 and Native APIs have coalesced over time. ↩︎

  2. This includes the Side-by-side assembly and SwitchBack compatability features. ↩︎

  3. Yosifovich, P. (2024). Windows Native API Programming. [online] Leanpub, pp.7–8. Available at: https://leanpub.com/windowsnativeapiprogramming. ↩︎

  4. When a native API returns an error, kernel32 code may call RtlNtStatusToDosError to map NTSTATUS to a Win32 ERROR, or just select an error. This error is passed to RtlRestoreLastWin32Error, which is just a wrapper around setting LastErrorValue in the Process Environment Block. ↩︎

As Zig has evolved, it has become a target to avoid calling Win32 APIs from `kernel32.dll` etc., instead using lower-level ones in `ntdll.dll`. This has been discussed in [#1840][gh], but this issue serves as its replacement, and a reformulation of why this practice is preferred. [gh]: https://github.com/ziglang/zig/issues/1840 Firstly: I am not a core Zig developer, and I do not speak for the team. I have made some pull requests in this area, and have helped out core developers to do the same. I am also not an expert on Windows InternalsTM, but I do read what other experts write and use their tools. --- On modern Windows, the Win32 APIs are implemented as a "Subsystem"[^ss] of the operating system, with its implementation relying on the "Native APIs" provided by the kernel. User-mode access to the Native API is done by calling functions exported by `ntdll`, which issues system calls to the NT kernel with the same arguments. [^ss]: Older versions of Windows NT had additional subsystems to support running POSIX and OS/2 applications, but these have since been removed. Aside from the introduction of WSL 1, the separation between Win32 and Native APIs have coalesced over time. Comparing the comprehensive [Win32][win32] API reference against the incidentally documented [Native][native] APIs, its clear which one Microsoft would prefer you use. The native API is treated as an implementation detail, whilst core parts of Windows' backwards compatibility strategy are implemented in Windows subsystem[^backcompat]. [win32]: https://learn.microsoft.com/en-us/windows/win32/api/ [native]: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/_kernel/ [sxs]: https://en.wikipedia.org/wiki/Side-by-side_assembly [sb]: https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-admx-appcompat#appcompatturnoffswitchback [^backcompat]: This includes the [Side-by-side assembly][sxs] and [SwitchBack][sb] compatability features. Why then would anyone use the Native API? The best answer I've found is in *Windows Native API Programming*[^win-native-pavel]: > Using the native APIs can have the following benefits: > - Performance - using the native API bypasses the standard Windows API, thus removing a software layer, speeding things up. > - Power - some capabilities are not provided by the standard Windows API, but are available with the native API. > - Dependencies - using the native API removes dependencies on subsystem DLLs, creating potentially smaller, leaner executables. > - Flexibility - in the early stages of Windows boot, native applications (those dependent on NtDll.dll only) can execute, while others cannot. > > That said, there are potential disadvantages to using the native API: > - It’s mostly undocumented, making it more difficult to use from the get go. This is where this book comes in! > - Being undocumented also means that Microsoft can make changes to the native API without warning, and no one can complain. That said, removal of capabilities from the native API or modification of existing functionality are rare. This is because some of Microsoft’s own tools and applications leverage the native API. [^win-native-pavel]: Yosifovich, P. (2024). Windows Native API Programming. [online] Leanpub, pp.7–8. Available at: https://leanpub.com/windowsnativeapiprogramming. The points on "performance" and "power" are more relevant for Zig for a couple reasons: - The backwards compatibility features come at the cost of unexpected (to the programmer) DLL loading, which in turn requires heap allocation, file loading and use of critical sections. These are things that Zig code [should handle by itself][why-zig], but a practical benefit is a reduction of memory/file errors that occur when the system is overloaded. This has been observed on the Windows CI machines. - Most Win32 APIs return a `BOOL` for success/failure, with the failure code needing to be fetched by calling `GetLastError`. Native APIs return an [`NTSTATUS`][ntstatus] directly, which is a richer set of error/informational codes. Using the native APIs avoids having to call into `kernel32`[^err] and allows for code to handle errors in the same `switch` where a function is called. - Speaking of types, the ones used for Native API programming are just better than the Win32 ones: - Times/timeouts are specified as `LARGE_INTEGER`s, a signed 64-bit time value with a 100ns resolution. An upgrade compared to the `dwMilliseconds` found in some Win32 functions, or `FILETIME`s which require bit-shifting. - `ACCESS_MASK`s, used when setting/querying rights on a kernel object, [map nicely][access-mask] to Zig's `packed struct`s. - Many Native APIs take an `IO_STATUS_BLOCK` (essentially one part of the Win32 `LPOVERLAPPED`), but these are often hidden by the Win32 APIs. - More features are exposed by the Native APIs than the Win32 APIs that call them. In addition, there are some replacement/extended APIs that are unused or have no equivalent: - `ReadDirectoryChangesW` is implemented using `NtNotifyChangeDirectoryFile`. On my Windows 11 machine, this is a thin wrapper around `NtNotifyChangeDirectoryFileEx`, which exposes more information via `FILE_NOTIFY_FULL_INFORMATION`. - [`NtFlushBuffersFileEx`][flush] allows for more granular flushing than `FlushBuffersFile`, and is used by e.g. [PostgreSQL][postgres] to provide the equivalent of `fdatasync`. [ntstatus]: https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/87fba13e-bf06-450e-83b1-9241dc81e781 [access-mask]: https://codeberg.org/ziglang/zig/src/commit/fcef9905ae859601d085576012b81dc05f67c46f/lib/std/os/windows.zig#L1240 [why-zig]: https://ziglang.org/learn/why_zig_rust_d_cpp/ [flush]: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-ntflushbuffersfileex [postgres]: https://doxygen.postgresql.org/win32fdatasync_8c_source.html#l00023 [^err]: When a native API returns an error, `kernel32` code may call `RtlNtStatusToDosError` to map `NTSTATUS` to a Win32 `ERROR`, or just select an error. This error is passed to `RtlRestoreLastWin32Error`, which is just a wrapper around setting `LastErrorValue` in the Process Environment Block. --- Does this mean that the Zig standard library will replace every Win32 API with the corresponding Native ones? *No, and don't expect it ever will*. There are certain parts of the Win32 API that are too complicated and/or undocumented enough that it's not worth the effort. These include: - ~~The console APIs, where the implementation was completely rewritten in Windows 8+ and remains under-researched.~~ Nevermind, [this happened](https://codeberg.org/ziglang/zig/pulls/31126) as I was writing this. - Bypassing the Winsock API and using `\Device\Afd` directly. - Anything to do with loading libraries. - Creating Win32 processes with `NtCreateUserProcess`. Heed this note from *Windows Native API Programming*: > There have been several attempts in the Infosec community to utilize `RtlCreateUserProcess` and/or `NtCreateUserProcess` to allow running Windows subsystem applications, with varying degrees of success. Part of the problem is the need to communicate with the Windows Subsystem process (`csrss.exe`) to notify it of the new process and thread. This turns out to be fragile, as different Windows versions may have somewhat different expectations. Everything else is fair game: - Some Win32 functions are ABI compatible with the Native ones, thus the implementation is a set of jumps from `kernel32.dll -> kernelbase.dll -> ntdll.dll -> ntoskrnl.exe`. [#25766][#25766] eliminated all of the remaining "forwarders", and you can see how e.g. `ReleaseSRWLockExclusive` forwards to `RtlReleaseSRWLockExclusive`. - Others are "thin wrappers" that call the native APIs and convert types/errors: - `CopySid` is just `RtlCopySid`, `NT_SUCCESS` to check the error code and `BaseSetLastNTError` for error mapping. - Some convert types like the Win32 `BOOL` (`i32`) into the Native `BOOLEAN` (`i8`) before calling the Native APIs. See `SetSystemTimeAdjustment`, `AddAuditAccessAce` for examples. - Some are "combination" wrappers that call multiple Native APIs that can be split into multiple parts: - `CreateIoCompletionPort` calls `NtCreateIoCompletion` if `ExistingCompletionPort` is `NULL`, then associates the `FileHandle` with the port and `CompletionKey` via `NtSetInformationFile`. - `SetHandleInformation` queries the status of `hObject` via `NtQueryObject` before calling `NtSetInformationObject`. - Finally, some functions setup the aforementioned compatibility features before calling Native APIs: - `ReadDirectoryChangesW` may create an [activation context][act-ctx] to load a Side-by-side assembly if `lpCompletionRoutine` is not `NULL`. - `GetOverlappedResult` always calls `SbSelectProcedure` if `bWait` is `TRUE`. This is part of the SwitchBack system introduced in Windows Vista. [#25766]: https://github.com/ziglang/zig/pull/25766 [act-ctx]: https://en.wikipedia.org/wiki/Side-by-side_assembly#Activation_contexts --- You, dear reader, may have some doubts as to whether this is all worth it. As someone who has done this work before, let me answer some common questions: > I use the functions in `std.os.windows.kernel32` directly, what will happen if I upgrade Zig and they are removed? If a re-implementation wasn't made, then a compile error most likely. The standard library should contain [only the host functions/types needed to support itself][reduceapi]. We recommend copying the necessary function definitions/types into your project, or using something like [zigwin32][zigwin32]. [reduceapi]: https://github.com/ziglang/zig/issues/4426 [zigwin32]: https://github.com/marlersoft/zigwin32 > Doing this breaks compatibility with older versions of Windows, which is important for me. True! But that is not (currently) a goal for Zig; standard library support is based on the earliest platform release supported by its developers. For Windows that means 10 and 11; and the server variants based on them. If *you* value backwards compatibility, replace calls to the standard library with Win32 ones instead. You can even replace your `main` function with `wWinMain`!. This practice is encouraged, as most of the standard library should work on free-standing environments. > Won't this get flagged by anti-virus scanners as suspicious? Unfortunately, [yes](https://github.com/ziglang/zig/issues/23392). We consider this a problem for the anti-virus scanners to solve. > Microsoft are free to change the Native API at will, and you will be left holding both pieces when things break. While this *can* happen, we have not (yet) been affected by any changes in the Win32 -> Native layers. Also, both API sets are quite stable and functionality is rarely removed. If anything we prefer using more modern Native APIs than older ones. That said, if there are issues then we [prefer to handle it in the standard library](https://github.com/ziglang/zig/pull/19738). Calling the Native APIs can bypass Windows-on-Windows support, but we have not seen significant issues by doing so. > I want to run my Zig programs on Wine. What if Wine's implementation differs from Windows, or is stubbed out? We consider this a bug in Wine, and not something to work around unless it significantly impacts us - e.g. CI failures on Linux due to running the test suite with Wine. The cases I know of involve the behaviour of [`NtQueryObject`][ntqueryobject] and the [console API][console] re: using the UTF-8 codepage. The former requires a [small workaround][workaround], the latter led to the standard library continuing to rely on `kernel32`. [ntqueryobject]: https://github.com/ziglang/zig/issues/26029 [console]: https://github.com/ziglang/zig/pull/14411 [workaround]: https://github.com/ziglang/zig/pull/17541/changes > Doing this requires observing the same implementation details as Windows/Wine/ReactOS. This is a fool's errand for a programming language. True! But that is our cross to bear. The standard library most definitely contains different logic re: path-name validation than Windows. This means that WoW64 path redirection most likely doesn't work when compiling for `x86-windows`. On the other hand, this allows the standard library to handle issues arising from [BadBatBut][badbatbut]. [badbatbut]: https://github.com/ziglang/zig/pull/19698 > Using undocumented APIs like this will just lead to runtime errors, SEH exceptions, and other nasal daemons. If some API is more sophisticated than a forwarder/wrapper, we investigate if the replacement is worth pursuing in the first case. We *prefer* the Native API, not *require* it; if you've encountered an issue due to this policy: **report it**! We are mostly reasonable people. But arguments like "Microsoft doesn't want you to do this" or "obviously this will lead to problems later on!" don't count. > Doing this means that Microsoft can't change/improve their internal APIs! While well-intentioned, this is incorrect for a couple reasons: - The design of the Native APIs are built to allow for future functionality. Instead of tightly coupling a structure to a system call as you often see for the \*Nixen, they are passed in as a tuple of `{OperationEnum, struct pointer, length of struct}`. For example, `NtQueryInformationFile` takes a `FILE_INFORMATION_CLASS` enum, and there has been [numerous additions][info-class] made over the many releases of Windows. The kernel also validates the structure length and will return an error if it is incorrect/too small. This can be used to allocate enough storage for flexible structures if the API takes a return length pointer. - If a new API is needed, the old one is never removed, instead: - The new API is exported via `ntdll`, and a superset of the old one with the addition of one-or-more parameters. You can distinguish them by their `Ex` suffix (or `Ex2` if the first time wasn't enough). The same thing happens in Win32 land where you have three `MapViewOfFile`s (four if you count the Numa variant). - The Win32 code is updated to call the new API, or the old API calls the new one. See the comment about `ReadDirectoryChangesW`. - Alternatively, the old code isn't changed at all and multiple Native APIs will live... *side-by-side*. - The existence of the Wine and ReactOS projects, along with Microsoft's own Side-by-side/Switchback systems, proves that has this *already happened*. There are large swathes of programs/libraries that depend on this functionality, and it is in everybody's best interest that they are kept available. [info-class]: https://ntdoc.m417z.com/file_information_class
Author
Contributor
Copy link

If you want to help out, here are some resources I and other contributors have used.

Code/Reverse Engineering Projects

  • The phnt headers: Probably the most up-to-date collection of types/structures/functions regarding the "Native API" i.e. ntdll.
  • ReactOS of course: Aims to be a complete re-implementation of Windows, covers the XP/Server 2003 era of native APIs, with some coverage of Windows Vista+.
  • Wine: Another implementation of the windows/win32 abis. While a good source, know that the project needs to be just correct enough to be able to run the vast majority of software, and may not be 1:1 accurate with modern Windows.

Useful Software

  • System Informer: The continuation of the Process Hacker project, has a lot of features for inspecting processes, services, network connections. It's also a waay better task manager than the windows 11 one.
  • The Sysinternals suite, which is a collection of tools for system introspection.
  • Pavel Yosifovich's tools: Another suite of apps from a windows internal expert.
  • NtObjectManager: A powershell module for interacting with the native api and creating types in a shell environment. Written by James Forshaw of Project Zero, whose blog posts are sometimes referenced in the standard library.
  • NtTrace: Traces calls to the native api ala strace.
  • dtrace for Windows: Official Windows port of dtrace. May require setup for desktop windows, but it was installed ootb on windows server core for me. Requires enabling dtrace support via bcdedit.

Literature

Finally, some nice literature:

  • Windows Internals, 7th Ed., Parts 1 & 2.
  • Windows Native API Programming by Pavel: Strong recommendation from me. Uses the phnt headers to document various parts of the native api with worked examples in C++.
  • Windows Security Internals by James Forshaw: Another great book. Uses NtObjectManager to explain/explore various parts of windows.
If you want to help out, here are some resources I and other contributors have used. ## Code/Reverse Engineering Projects - The [phnt headers](https://github.com/winsiderss/systeminformer/tree/master/phnt): Probably the most up-to-date collection of types/structures/functions regarding the "Native API" i.e. ntdll. - [ReactOS](https://github.com/reactos/reactos) of course: Aims to be a complete re-implementation of Windows, covers the XP/Server 2003 era of native APIs, with some coverage of Windows Vista+. - [Wine](https://gitlab.winehq.org/wine/wine): Another implementation of the windows/win32 abis. While a good source, know that the project needs to be just correct enough to be able to run the vast majority of software, and may not be 1:1 accurate with modern Windows. ## Useful Software - [System Informer](https://systeminformer.sourceforge.io/): The continuation of the Process Hacker project, has a lot of features for inspecting processes, services, network connections. It's also a *waay* better task manager than the windows 11 one. - The [Sysinternals suite](https://learn.microsoft.com/en-us/sysinternals/), which is a collection of tools for system introspection. - Pavel Yosifovich's [tools](https://github.com/zodiacon/AllTools): Another suite of apps from a windows internal expert. - [NtObjectManager](https://www.powershellgallery.com/packages/NtObjectManager): A powershell module for interacting with the native api and creating types in a shell environment. Written by James Forshaw of Project Zero, whose blog posts are sometimes referenced in the standard library. - [NtTrace](https://github.com/rogerorr/NtTrace): Traces calls to the native api ala strace. - [dtrace for Windows](https://learn.microsoft.com/en-us/windows-hardware/drivers/devtest/dtrace): Official Windows port of dtrace. May require setup for desktop windows, but it was installed ootb on windows server core for me. Requires enabling dtrace support via bcdedit. ## Literature Finally, some nice literature: - Windows Internals, 7th Ed., Parts 1 & 2. - [Windows Native API Programming](https://leanpub.com/windowsnativeapiprogramming) by Pavel: Strong recommendation from me. Uses the phnt headers to document various parts of the native api with worked examples in C++. - [Windows Security Internals](https://nostarch.com/windows-security-internals) by James Forshaw: Another great book. Uses NtObjectManager to explain/explore various parts of windows.

If you value backwards compatibility, replace calls to the standard library with Win32 ones instead.

This is a perfectly reasonable strategy but it has the downside of giving up the Zig ecosystem. I would instead encourage developers to create and maintain an Io implementation that has the goal of supporting old Windows versions. This way you can continue to take advantage of, and contribute to, the Zig ecosystem.

Microsoft are free to change the Native API at will, and you will be left holding both pieces when things break.

In addition to what @The-King-of-Toasters said, the worst case scenario is really mild: A new version of windows comes out, breaking ntdll compatibility. Zig project adds a fix to the std lib. Application developer recompiles their zig project from source, and ships an update to their users.

I want to run my Zig programs on Wine. What if Wine's implementation differs from Windows, or is stubbed out?

In addition to what @The-King-of-Toasters said, we collaborate with Wine upstream to get these issues fixed. For example:

> If you value backwards compatibility, replace calls to the standard library with Win32 ones instead. This is a perfectly reasonable strategy but it has the downside of giving up the Zig ecosystem. I would instead encourage developers to create and maintain an Io implementation that has the goal of supporting old Windows versions. This way you can continue to take advantage of, and contribute to, the Zig ecosystem. > Microsoft are free to change the Native API at will, and you will be left holding both pieces when things break. In addition to what @The-King-of-Toasters said, the worst case scenario is really mild: A new version of windows comes out, breaking ntdll compatibility. Zig project adds a fix to the std lib. Application developer recompiles their zig project from source, and ships an update to their users. > I want to run my Zig programs on Wine. What if Wine's implementation differs from Windows, or is stubbed out? In addition to what @The-King-of-Toasters said, we collaborate with Wine upstream to get these issues fixed. For example: * https://bugs.winehq.org/show_bug.cgi?id=47979 * https://bugs.winehq.org/show_bug.cgi?id=59126

@andrewrk wrote in #31131 (comment):

Microsoft are free to change the Native API at will, and you will be left holding both pieces when things break.

In addition to what @The-King-of-Toasters said, the worst case scenario is really mild: A new version of windows comes out, breaking ntdll compatibility. Zig project adds a fix to the std lib. Application developer recompiles their zig project from source, and ships an update to their users.

Only works if the software is still supported. And considering that over 80% of all video games ever developed (and 70% of the ones developed for Windows; and Zig will likely be used for them), this would accelerate this number.

And while there are people who actively patch the binaries if they care about them, that's rare.

@andrewrk wrote in https://codeberg.org/ziglang/zig/issues/31131#issuecomment-10395612: > > Microsoft are free to change the Native API at will, and you will be left holding both pieces when things break. > > In addition to what @The-King-of-Toasters said, the worst case scenario is really mild: A new version of windows comes out, breaking ntdll compatibility. Zig project adds a fix to the std lib. Application developer recompiles their zig project from source, and ships an update to their users. Only works if the software is still supported. And considering that over 80% of all video games ever developed (and 70% of the ones developed for Windows; and Zig will likely be used for them), this would accelerate this number. And while there are people who actively patch the binaries if they care about them, that's rare.
Owner
Copy link

There are certain parts of the Win32 API that are too complicated and/or undocumented enough that it's not worth the effort. These include:

  • [...]
  • Bypassing the Winsock API and using \Device\Afd directly.

I'm curious what the rationale was behind this particular point about ws2 vs \Device\Afd, because it seems to me like the Winsock API is an absolute mess which leads me to think that it'd probably be better in terms of code complexity (note that complexity != number of lines) and efficiency to use the NT device object directly. Has this interface been unstable historically, or is it very difficult to use, or was there some other reason for this advice?

> There are certain parts of the Win32 API that are too complicated and/or undocumented enough that it's not worth the effort. These include: > * [...] > * Bypassing the Winsock API and using `\Device\Afd` directly. I'm curious what the rationale was behind this particular point about ws2 vs `\Device\Afd`, because it seems to me like the Winsock API is an absolute mess which leads me to think that it'd probably be better in terms of code complexity (note that complexity != number of lines) and efficiency to use the NT device object directly. Has this interface been unstable historically, or is it very difficult to use, or was there some other reason for this advice?
Author
Contributor
Copy link

Some projects like c-ares and wepoll drop down to \Device\Afd to get epoll-like semantics on Windows. Other than ws2_32, the only project that uses it directly is midipix, which implements its own sockets layer. AFAIK it maintains two versions of functions like socket, connect, bind to support pre and post Windows Vista versions.

Some projects like [c-ares] and [wepoll] drop down to `\Device\Afd` to get epoll-like semantics on Windows. Other than `ws2_32`, the only project that uses it directly is midipix, which implements its [own sockets layer][midipix]. AFAIK it maintains two versions of functions like `socket`, `connect`, `bind` to support pre and post Windows Vista versions. [c-ares]: https://github.com/c-ares/c-ares/blob/2870f6b14723b3bb77a365f4745abcbd74c2ffad/src/lib/event/ares_event_win32.c#L48 [wepoll]: https://github.com/piscisaureus/wepoll [midipix]: https://dev.midipix.org/runtime/ntapi/blob/main/f/src/socket

Does this mean that the Zig standard library will replace every Win32 API with the corresponding Native ones? No, and don't expect it ever will. There are certain parts of the Win32 API that are too complicated and/or undocumented enough that it's not worth the effort.

Since I am currently working on getting the hostname on Windows: Where does getting a value from the Windows Registry lie on that scale?

Because I have the choice between doing one Win32 API call which does this for me or asking the Windows registry via the Nt API (which is kinda confusing to me since I never really interfaced with that).

Also, in case it's the Nt API, since I would need to add new structs to std.os.windows.ntdll, how does deal with a struct with a VLA member? e.g. KEY_VALUE_PARTIAL_INFORMATION

> Does this mean that the Zig standard library will replace every Win32 API with the corresponding Native ones? No, and don't expect it ever will. There are certain parts of the Win32 API that are too complicated and/or undocumented enough that it's not worth the effort. Since I am currently working on getting the hostname on Windows: Where does getting a value from the Windows Registry lie on that scale? Because I have the choice between doing one Win32 API call which does this for me or asking the Windows registry via the Nt API (which is kinda confusing to me since I never really interfaced with that). Also, in case it's the Nt API, since I would need to add new structs to `std.os.windows.ntdll`, how does deal with a struct with a VLA member? e.g. [KEY_VALUE_PARTIAL_INFORMATION](https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_key_value_partial_information)

@KilianHanich see #31384 for where the last remaining registry-related advapi32 bindings were removed in favor of ntdll APIs (and the new implementation happens to deal with KEY_VALUE_PARTIAL_INFORMATION).

Your questions seem potentially off-topic for the issue tracker, but I'd be happy to answer them if you ping/DM me on IRC/Discord/the ZSF Zulip (info here)

@KilianHanich see https://codeberg.org/ziglang/zig/pulls/31384 for where the last remaining registry-related advapi32 bindings were removed in favor of ntdll APIs (and the new implementation happens to deal with `KEY_VALUE_PARTIAL_INFORMATION`). Your questions seem potentially off-topic for the issue tracker, but I'd be happy to answer them if you ping/DM me on IRC/Discord/the ZSF Zulip ([info here](https://ziglang.org/community/))
Contributor
Copy link

https://ntdoc.m417z.com/
very handy way to browse phnt headers structures and function prototypes as described above phnt and Wine are the most complete and up to date sources.

https://www.vergiliusproject.com/kernels/x64
very ambitious project that focuses on structures including major Windows releases and their subversions from Windows XP up to Windows 25H2.

https://ntdoc.m417z.com/ very handy way to browse phnt headers structures and function prototypes as described above phnt and Wine are the most complete and up to date sources. https://www.vergiliusproject.com/kernels/x64 very ambitious project that focuses on structures including major Windows releases and their subversions from Windows XP up to Windows 25H2.

As of 9ac1386c10, the only remaining non-ntdll extern functions in the std lib are:

andy@bark ~/s/zig (master)> grep -RI 'pub extern "' lib/std/os/windows/ | grep -v ntdll
lib/std/os/windows/kernel32.zig:pub extern "kernel32" fn CreateProcessW(
lib/std/os/windows/crypt32.zig:pub extern "crypt32" fn CertOpenSystemStoreW(
lib/std/os/windows/crypt32.zig:pub extern "crypt32" fn CertCloseStore(
lib/std/os/windows/crypt32.zig:pub extern "crypt32" fn CertEnumCertificatesInStore(

This issue is completed.

As of 9ac1386c10736bc249f3891f34f23424531917a5, the only remaining non-ntdll extern functions in the std lib are: ``` andy@bark ~/s/zig (master)> grep -RI 'pub extern "' lib/std/os/windows/ | grep -v ntdll lib/std/os/windows/kernel32.zig:pub extern "kernel32" fn CreateProcessW( lib/std/os/windows/crypt32.zig:pub extern "crypt32" fn CertOpenSystemStoreW( lib/std/os/windows/crypt32.zig:pub extern "crypt32" fn CertCloseStore( lib/std/os/windows/crypt32.zig:pub extern "crypt32" fn CertEnumCertificatesInStore( ``` This issue is completed.
alexrp modified the milestone from Upcoming to 0.16.0 2026年03月23日 21:15:51 +01:00
Sign in to join this conversation.
No Branch/Tag specified
master
elfv2
spork8
restricted
0.16.x
panic-rewrite
parking-futex-lockfree
windows-Io-cleanup
Io-watch
ParseCommandLineOptions
windows-async-files
poll-ring
debug-file-leaks-differently
debug-file-leaks
ProcessPrng
elfv2-dyn
jobserver
threadtheft
io-threaded-no-queue
0.15.x
Io.net
comptime-allocator
restricted-function-pointers
cli
wasm-linker-writer
wrangle-writer-buffering
sha1-stream
async-await-demo
fixes
0.14.x
ast-node-methods
macos-debug-info
make-vs-configure
fuzz-macos
sans-aro
ArrayList-reserve
incr-bug
llvm-ir-nosanitize-metadata
ci-tarballs
ci-scripts
threadpool
0.12.x
new-pkg-hash
json-diagnostics
more-doctests
rework-comptime-mutation
0.11.x
ci-perf-comment
stage2-async
0.10.x
autofix
0.9.x
aro
hcs
0.8.x
0.7.x
0.16.0
0.15.2
0.15.1
0.15.0
0.14.1
0.14.0
0.12.1
0.13.0
0.12.0
0.11.0
0.10.1
0.10.0
0.9.1
0.9.0
0.8.1
0.8.0
0.7.1
0.7.0
0.6.0
0.5.0
0.4.0
0.3.0
0.2.0
0.1.1
0.1.0
Labels
Clear labels
abi/f32
abi/ilp32
abi/sf
accepted
This proposal is planned.
arch/21k
arch/6502
arch/aarch64
arch/alpha
arch/amdgcn
arch/arc
arch/arc32
arch/arc64
arch/arm
arch/avr
arch/bfin
arch/bpf
arch/colossus
arch/cris
arch/csky
arch/dlx
arch/epiphany
arch/fr30
arch/frv
arch/hexagon
arch/hppa
arch/hppa64
arch/ia64
arch/kalimba
arch/kvx
arch/lanai
arch/lm32
arch/loongarch32
arch/loongarch64
arch/m32r
arch/m68k
arch/m88k
arch/mcore
arch/microblaze
arch/mips
arch/mips64
arch/mmix
arch/moxie
arch/mrisc32
arch/msp430
arch/nds32
arch/ns32k
arch/nvptx
arch/or1k
arch/powerpc
arch/powerpc64
arch/propeller
arch/riscv32
arch/riscv64
arch/rl78
arch/rx
arch/s390x
arch/sh
arch/sparc
arch/sparc64
arch/spirv
arch/spu
arch/tricore
arch/v850
arch/vax
arch/vc4
arch/ve
arch/wasm
arch/x86
arch/x86_64
arch/xcore
arch/xtensa
autodoc
The web application for interactive documentation and generation of its assets.
backend/c
The C backend outputs C source code.
backend/llvm
The LLVM backend outputs an LLVM bitcode module.
backend/self-hosted
The self-hosted backends produce machine code directly.
binutils
Zig's included binary utilities: zig ar, zig dlltool, zig lib, zig ranlib, zig objcopy, and zig rc.
breaking
Implementing this issue could cause existing code to no longer compile or have different behavior.
build system
The Zig build system - zig build, std.Build, the build runner, and package management.
debug info
An issue related to debug information (e.g. DWARF) produced by the Zig compiler.
docs
An issue with documentation, e.g. the language reference or standard library doc comments.
error message
This issue points out an error message that is unhelpful and should be improved.
frontend
Tokenization, parsing, AstGen, ZonGen, Sema, Legalize, and Liveness.
fuzzing
An issue related to Zig's integrated fuzz testing.
incremental
Reuse of internal compiler state for faster compilation.
lib/c
This issue relates to Zig's libc implementation and/or vendored libcs.
lib/compiler-rt
This issue relates to Zig's compiler-rt library.
lib/cxx
This issue relates to Zig's vendored libc++ and/or libc++abi.
lib/std
This issue relates to Zig's standard library.
lib/tsan
This issue relates to Zig's vendored libtsan.
lib/ubsan-rt
This issue relates to Zig's ubsan-rt library.
lib/unwind
This issue relates to Zig's vendored libunwind.
linking
Zig's integrated object file and incremental linker.
miscompilation
The compiler reports success but produces semantically incorrect code.
os/android
os/contiki
os/dragonfly
os/driverkit
os/emscripten
os/freebsd
os/fuchsia
os/haiku
os/hermit
os/hurd
os/illumos
os/ios
os/linux
os/maccatalyst
os/macos
os/managarm
os/netbsd
os/ohos
os/openbsd
os/plan9
os/redox
os/rtems
os/serenity
os/tvos
os/uefi
os/visionos
os/wasi
os/watchos
os/windows
proposal
This issue suggests language modifications. If it also has the "accepted" label then it is planned.
release notes
This issue or pull request should be mentioned in the release notes.
testing
This issue is related to testing the compiler, standard library, or other parts of Zig.
zig cc
Zig as a drop-in C-family compiler.
zig fmt
The Zig source code formatter.
zig reduce
The Zig source code reduction tool.
bounty
https://ziglang.org/news/announcing-donor-bounties
bug
Observed behavior contradicts documented or intended behavior.
contributor-friendly
This issue is limited in scope and/or knowledge of project internals.
downstream
An issue with a third-party project that uses this project.
enhancement
Solving this issue will likely involve adding new logic or components to the codebase.
infra
An issue related to project infrastructure, e.g. continuous integration.
optimization
A task to improve performance and/or resource usage.
question
No questions on the issue tracker; use a community space instead.
regression
Something that used to work in a previous version stopped working
upstream
An issue with a third-party project that this project uses.
use case
Describes a real use case that is difficult or impossible, but does not propose a solution.
No labels
abi/f32
abi/ilp32
abi/sf
accepted
arch/21k
arch/6502
arch/aarch64
arch/alpha
arch/amdgcn
arch/arc
arch/arc32
arch/arc64
arch/arm
arch/avr
arch/bfin
arch/bpf
arch/colossus
arch/cris
arch/csky
arch/dlx
arch/epiphany
arch/fr30
arch/frv
arch/hexagon
arch/hppa
arch/hppa64
arch/ia64
arch/kalimba
arch/kvx
arch/lanai
arch/lm32
arch/loongarch32
arch/loongarch64
arch/m32r
arch/m68k
arch/m88k
arch/mcore
arch/microblaze
arch/mips
arch/mips64
arch/mmix
arch/moxie
arch/mrisc32
arch/msp430
arch/nds32
arch/ns32k
arch/nvptx
arch/or1k
arch/powerpc
arch/powerpc64
arch/propeller
arch/riscv32
arch/riscv64
arch/rl78
arch/rx
arch/s390x
arch/sh
arch/sparc
arch/sparc64
arch/spirv
arch/spu
arch/tricore
arch/v850
arch/vax
arch/vc4
arch/ve
arch/wasm
arch/x86
arch/x86_64
arch/xcore
arch/xtensa
autodoc
backend/c
backend/llvm
backend/self-hosted
binutils
breaking
build system
debug info
docs
error message
frontend
fuzzing
incremental
lib/c
lib/compiler-rt
lib/cxx
lib/std
lib/tsan
lib/ubsan-rt
lib/unwind
linking
miscompilation
os/android
os/contiki
os/dragonfly
os/driverkit
os/emscripten
os/freebsd
os/fuchsia
os/haiku
os/hermit
os/hurd
os/illumos
os/ios
os/linux
os/maccatalyst
os/macos
os/managarm
os/netbsd
os/ohos
os/openbsd
os/plan9
os/redox
os/rtems
os/serenity
os/tvos
os/uefi
os/visionos
os/wasi
os/watchos
os/windows
proposal
release notes
testing
zig cc
zig fmt
zig reduce
bounty
bug
contributor-friendly
downstream
enhancement
infra
optimization
question
regression
upstream
use case
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ziglang/zig#31131
Reference in a new issue
ziglang/zig
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?