-
Notifications
You must be signed in to change notification settings - Fork 7.9k
[RFC] Add support for building to the wasm32-wasi target #10457
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This adds support for building php-cgi
to the wasm32-wasi platform.
In order to build in your own environment, you can use wasi-sdk
directly, or use a build container image to make the build process easier. wasi-sdk
is a project that makes it easier to build C projects to WebAssembly + WASI. It bundles clang
and the wasi-libc
project.
As an improvement, we can update the configure.ac
to accept a --wasi-sdk
argument, akin to what the sqlite project is going to do in order to support WebAssembly + WASI.
Building
You can build either by using wasi-sdk
directly, or by using a container image we have prepared that contains everything required.
Build with wasi-sdk
Install wasi-sdk
first:
$ wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-19/wasi-sdk-19.0-linux.tar.gz
$ tar xvf wasi-sdk-19.0-linux.tar.gz
$ export WASI_SDK_PATH=`pwd`/wasi-sdk-19.0
$ export CC="${WASI_SDK_PATH}/bin/clang --sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"
$ export LD="${WASI_SDK_PATH}/bin/wasm-ld"
Set common flags:
$ export CFLAGS="-D_POSIX_SOURCE=1 -D_GNU_SOURCE=1 -DHAVE_FORK=0 -D_WASI_EMULATED_GETPID -D_WASI_EMULATED_SIGNAL -D_WASI_EMULATED_PROCESS_CLOCKS"
$ export LDFLAGS="-lwasi-emulated-getpid -lwasi-emulated-signal -lwasi-emulated-process-clocks"
Configure:
$ ./buildconf --force
$ ./configure --host=wasm32-wasi host_alias=wasm32-musl-wasi --target=wasm32-wasi target_alias=wasm32-musl-wasi --without-libxml --disable-dom --without-iconv --without-openssl --disable-simplexml --disable-xml --disable-xmlreader --disable-xmlwriter --without-pear --disable-phar --disable-opcache --disable-zend-signals --without-pcre-jit --with-sqlite3 --enable-pdo --with-pdo-sqlite --disable-fiber-asm --without-sqlite3
And build!, make -j cgi
.
You can find the binary under sapi/cgi/php-cgi
as usual.
Notes and automated build artifacts
In this example we have only built the CGI binary, but the CLI binary can also be adapted to build for wasm32-wasi.
At the webassembly-language-runtimes project we are continuously releasing PHP binaries. You can find released artifacts here, and on every commit of the repo.
These artifacts are also linked against a sqlite3 build ported to WASI -- the sqlite3 project will officially release the WASI version of sqlite3. --
Running the binaries
You can download any WebAssembly runtime compatible with WASI (most of them). For example, wasmtime
:
$ cat hello-world.php
<?php
echo "Hello, world!";
?>
$ wasmtime --mapdir .::. php-cgi.wasm -- hello-world.php
X-Powered-By: PHP/8.3.0-dev
Content-type: text/html; charset=UTF-8
Hello, world!
It's required to map the current directory, so that the hello-world.php
is found by the WebAssembly module within the sandbox.
Tests
Running the test suite in our automation is still work in progress. We will improve this changes so that all test that are relevant to the wasm32-wasi platform are green.
setjmp/longjmp emulation
We have a contribution that enables the setjmp/longjmp emulation in the same way as the Ruby interpreter has done. This contribution is still to be merged in our own repository, but we are also excited to integrate it on the changes we aim to upstream.
Similarly, we also have work in progress that enables PHP fibers using the same Asyncify strategy.
Feedback
We are looking forward for feedback on these changes. We want to contribute this changes upstream, and are willing to update this contribution so that it fits your needs and requirements.
Please, let us know your thoughts, or anything you might need in order for this change to land.
Thank you!
Further context: we have successfully run wordpress in the browser, and also in the server side with Apache and mod_wasm.
b726faa
to
a94519d
Compare
Please add a CI workflow for WASM, see https://github.com/php/php-src/blob/master/.github/workflows/push.yml
a94519d
to
de67bf7
Compare
de67bf7
to
000954b
Compare
Please add a CI workflow for WASM, see https://github.com/php/php-src/blob/master/.github/workflows/push.yml
Thanks @mvorisek. Updated PR. Current result as of now:
Number of tests : 379 334
Tests skipped : 45 ( 11.9%) --------
Tests warned : 0 ( 0.0%) ( 0.0%)
Tests failed : 17 ( 4.5%) ( 5.1%)
Tests passed : 317 ( 83.6%) ( 94.9%)
---------------------------------------------------------------------
Time taken : 125 seconds
Can somebody allow GH actions to execute on this PR?
000954b
to
d2fada3
Compare
8ce4db8
to
e033a5a
Compare
38b96cf
to
4584dc2
Compare
Thanks a lot for the reviews. I have updated the PR.
Let me know if by the compat layer you meant something specifically different than what this patch provides. I looked at the win32 compat layer, but I also see many ifdefs here and there on the source code, so I tried to update the patch from scratch in a similar form, while trying not to be very intrusive with the current codebase. Please, let me know if you have a specific example that would look better on the current codebase, I would be happy to adapt.
The CI result with the wasi execution can be found at https://github.com/ereslibre/php-src/actions/runs/4403303495
There are a lot of segfaults in the CI: https://github.com/ereslibre/php-src/actions/runs/4403303495/jobs/7711483600#step:7:9775
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it tries to disable way too many function. As I understand wasm is missing some of libc functions, right? If so, then it would be probably better to wrap the missing pieces with functions that have the same signature like libc functions and always fail. Then you can link those for wasm and it won't be necessary to add that many ifdefs.
Zend/zend.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This might need some more checking of places where the bailout is used and veryfying if it might have any impact on the apllication under wasm. We basically don't want to introduce something that will result to future bug and I would imagine that disabling bailout might be one of those things without a proper checking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, absolutely. Also, we will implement the bailout logic once we include setjmp/longjmp support.
main/fastcgi.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's the actual point of using FCGI if it's not re-using connection (if it's 0, then it closes the connection)? Why don't you just use plain CGI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the patch; fastcgi.c
requires some changes because cgi_main
uses certain symbols that come from fastcgi.c
; and we are interested in php-cgi
.
wasi/main/php_open_temporary_file.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use tabs and not spaces (this is not the place where you indent with spaces so please all other cases as well)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I am going to check everything, my editor is pretty stubborn at times.
4584dc2
to
5d77869
Compare
5d77869
to
32a7f81
Compare
f25535d
to
0188a82
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just to make the CI job name shorter/nicer...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would keep WASM32_WASI
if that's fine, given that WASM64 be added eventually
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All core tests (tests that does not require any unsupported extension) must pass or they must be exluded if they are expected to fail ATM.
setjmp/longjmp emulation
Fibers are core part of the PHP language since PHP 8.0 release. Can the support be added from the day zero?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All core tests (tests that does not require any unsupported extension) must pass or they must be exluded if they are expected to fail ATM.
Will exclude tests that don't apply on this platform in a next pass.
Fibers are core part of the PHP language since PHP 8.0 release. Can the support be added from the day zero?
We should look into that. If it's a prerequisite for this patch to land, then yes. If it's fine to iterate I'd prefer so, to not block the whole patch on smaller specifics, given the patch is already somewhat big.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't actually finished the review. But the trend seemed to continue. I think we need a better approach for missing functions/features than stubbing them out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason why this is in its own file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea was to be as less intrusive as possible with the existing tree, avoiding unnecessary ifdefs that might clutter existing files.
If you think there's a better way I'm fine with any proposal!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this need to be in a separate file? Separate files are a bit of a pain with autoconf because it require calling ./configure
again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, same rationale as #10457 (comment), to be as less intrusive as possible.
I understand the ./configure
re-run concern, but at this stage I think is better to keep them well separated, and as WASI evolves with future snapshots start merging with the main tree as things establish on upcoming wasi-libc versions.
060ea6f
to
b7883f5
Compare
This change performs initial adaptations allowing the PHP interpreter to be compiled for the wasm32-wasi target. Co-Authored-By: Asen Alexandrov <alexandrov@vmware.com>
WebAssembly + WASI GH actions have specific tasks for the following steps: - apt: an existing PHP installation will be used as the test runner, as opposed to the binary being built. It also installs other dependencies used by later stages, such as Wasmtime (WebAssembly runtime), and Binaryen (contains `wasm-opt`, to be used on the optimization phase.) - configure-wasi: specific configure values. - optimize-wasi (new action): performs binaryen optimization passes on the resulting binary. - test-wasi: requires a wrapper that will use `wasmtime` to run the `php` CLI compiled to WebAssembly.
b7883f5
to
908a7a7
Compare
Thanks for your review @iluuu1994. I have updated the patch fixing all your comments, and I also commented about the separate files here: #10457 (comment).
This change will evolve as the wasi-libc evolves. However, the principle is always to be as less intrusive as possible with the current codebase. When there are concepts that are not directly mappable to WebAssembly/WASI, we stubbed them out; in this patch this happened in a case by case basis, so please, let us know if you'd like to see things handled in a different way.
The goal is to propose this patch for landing on the PHP tree, which I think might need an RFC process, but before that happens I think is worth getting done as much as possible. Because of this, feedback from PHP contributors is highly valuable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are still plenty of places where functions just "appear" to exist, but without doing anything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Magic numbers are not a good idea. If WASM doesn't define LOG_NOTICE, then have a block somewhere else that defines this (and I suspect other constants) in one go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reads really odd - if WASI is active, you only use a forward declaration. I would rather do it like:
#ifndef PHP_WASI
static void php_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS) /* {{{
{
...
}
#else
static void php_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS) /* {{{
{
php_wasi_shell_exec_ex(INTERNAL_FUNCTION_PARAMETERS_PASSTHRU);
}
#endif // PHP_WASI
And then have your new files implement the php_wasi_*
variant(s).
But considering that doesn't do anything on WASI, this function should also just not even exist on that platform.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, these functions do nothing, so they shouldn't exist.
I thank everyone who has provided feedback on this PR. I'm closing it as is. If there is interest in reviving this work I encourage folks to take the lead on that.
Uh oh!
There was an error while loading. Please reload this page.
Add support for building to the wasm32-wasi target
This change performs initial adaptations allowing the PHP interpreter
to be compiled for the wasm32-wasi target.
Co-Authored-By: Asen Alexandrov alexandrov@vmware.com