-
Notifications
You must be signed in to change notification settings - Fork 8k
Ninja support on Windows #17445
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ninja support on Windows #17445
Conversation
win32/build/build.ninja
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For an English VS, just remove that line. For other languages, see https://ninja-build.org/manual.html#_deps.
I don't really have an opinion on this, I barely care about Windows support.
Regarding the functionality, this is now close to what I had in mind: to be able to do full snapshot builds (--enable-cli-win32 and --enable-phpdbgs are not supported, but these are rarely needed, and the implementation is hackish anyway).
I've already mentioned the performance improvements for debug builds above, and that can also been seen in CI (https://github.com/php/php-src/actions/runs/12734135483/job/35491329533 vs https://github.com/php/php-src/actions/runs/12734364142/job/35491832165). However, Ninja can also be used for clang builds which are in serious need for speed (a minimal release build takes ~5min here with NMake; less than 1.5min with Ninja).
Having had to build on Windows occasionally, I agree that nmake is not great. Ninja seems much better, but I have no experience with it except indirectly when building projects that use it.
Switching to CMake would bring Ninja support, but we are far from this, so switching directly to Ninja on Windows seems reasonable, especially since we don't have to re-write or duplicate all the config.w32 files. Maybe we can eventually remove nmake support, so that we can remove the Makefile.frag.w32 files.
Maybe we can eventually remove nmake support, so that we can remove the Makefile.frag.w32 files.
I have doubts that it's worth the effort. So far, I have not spent any time on phpize builds (these are usually pretty fast anyway), and if you want debugging or other tooling (ASan), you're better of doing in-tree (or "pecl-tree") builds. And there are some many more cases that would need to be ported to Ninja, e.g. nmake run ARGS=... DEBUGGER=1, and more involved all the packaging stuff (nmake snap). And it might be possible that some external extension use complex Makefile.frag.w32 files for whatever reason (porting these might be almost impossible; Ninja is pretty restricted).
My main interest is in having a speed-up for actual debug builds and development in general, hopefully with minimally invasive changes to the build chain, and retaining full compatibility with NMake (currently, after configure you can just run ninja, but follow up with the typical nmake test etc.)
Switching to CMake would bring Ninja support, but we are far from this, [...]
Unfortunately! I would highly appreciate a unified build system, and even thought about switching to autotools on Windows, what is likely possible, but would be painfully slow (even configure might run for minutes; try with Cygwin or MinGW).
I don't build that often on Windows and usually doing the minimal builds which is not that slow but if it makes things quicker and better for you, then it makes sense. I don't know much about ninja except using it on some occassions so don't have that much input about it either.
Might be worth to hear from @petk as he is progressing with his CMake PHP build by the look of it so he might have some valuable input here as well.
I've filed petk/php-build-system#32. Let's see where that goes. :)
9313dd4 to
748d70a
Compare
748d70a to
241ca52
Compare
* We also switch to /Z7 to avoid some build flakyness. * Support ninja.frag.w32 Unfortunately, that requires a hard-coded hack regarding OPcache JIT, since Ninja does not support multiple build lines to specify dependencies. * Explicitly require Ninja >= 1.7
241ca52 to
e54854a
Compare
I'm closing this PR for now, since
- I still haven't a good idea how to clean up the mess/make it maintable (would likely need to start with the Makefile generation, but that may not make sense if we switch to CMake in the not too distant future)
- I had to switch to
/Z7since there have been serious issues with concurrent PDB writing (/FShelped, but didn't solve all issues; possibly I've forgot some cases – possibly/FSonly works properly for multi-source file invocations of cl.exe) - I've also employed a couple of hacks (e.g. for JIT), and have no idea how to resolve that cleanly
- there doesn't seem to be much interest from others, and for me it's simpler to keep it up to date in a GH branch, but I don't want to trigger CI everytime I rebase (and don't like to work around that)
Note that I'm using Ninja builds most of the time, and I'm planning to keep on doing so (DX improvement is amazing), so I won't forget about it. Anybody is welcome to check out https://github.com/cmb69/php-src/tree/cmb/ninja for themselves.
Uh oh!
There was an error while loading. Please reload this page.
Contrary to many other make implementations, NMake does not support parallel builds. Instead, MS enabled cl.exe for parallel compilation (
/MP), but that is not allowed for debug builds, likely due to the documented limitation:This implies that debug builds are particularly slow (on my machine, unoptimized debug builds are only minimally faster than fully optimized release builds; they are likely slower on more powerful machines).
Now Visual Studio ships Ninja as part of its CMake support, and Ninja supports parallel builds. Thus I've hacked support for generating a build.ninja into our Windows build tooling; just enough to be able to do a minimal build (
configure --disable-all --enable-cli --enable-debug), and was pleasantly surprised that this improved build time by a factor of three (again, I expect even more impressive results with a more powerful machine). Release builds are also slightly faster (about 10% percent).Since Ninja supports header dependencies almost fully automatically, I've added support for that as well, so it's no longer necessary to take care of that yourself (i.e. when making a change to a header,
nmakewould do nothing; you need to touch relevant files, and for changes to a Zend header, you usually need tonmake clean && nmake– terrible UX). This works currently for German VS installations only (see comment below).Another benefit of using Ninja instead of NMake is the console output during building. While our NMake output is pretty verbose for debug builds, Ninja is terse, usually only showing one line of progress, and only in case of a build failure showing the whole command that failed.
To round this up, there is still of lot of work to be done, but before doing that, I would like to gauge interest in Ninja support – and maybe there are already other alternatives readily available.
Maybe @nielsdos, @arnaud-lb and @bukka are interested in this.
TODO:
msvc_deps_prefix)--enable-ninjaconfigure flag