-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Building arrow for webR #49121
Unanswered
arnaud-feldmann
asked this question in
Q&A
Building arrow for webR
#49121
Hi,
As I've seen that arrow can be used with pyodide, I tried to build it for webR within a mini-cran repo with the standard GH.
https://github.com/arnaud-feldmann/build_rwasm_repo/tree/20e01e9cac3970d2634593daa95f8f4c5f7890e1
I followed the emscripten compilation here https://arrow.apache.org/docs/developers/cpp/emscripten.html to provide the webr docker image with arrow, and then I compiled the arrow packages.
That is, the Dockerfile is :
FROM ghcr.io/r-wasm/webr:v0.5.8
RUN git clone --depth 1 --branch apache-arrow-23.0.0 https://github.com/apache/arrow.git /tmp/arrow && \
cd /tmp/arrow/cpp && \
emcmake cmake --preset "ninja-release-emscripten" && \
ninja install && \
rm -rf /tmp/arrow
Within this image i try to compile my packages with the GH provided by webR. The compilation works.
As for arrow, during the compilation, everything seems to be ok and everything seems to be static linked with emscripten :
*** Trying Arrow C++ found by pkg-config: /opt/emsdk/upstream/emscripten/cache/sysroot
**** C++ library version 23.0.0 is supported by R version 23.0.0
PKG_CFLAGS=-I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON
PKG_LIBS=-L/opt/emsdk/upstream/emscripten/cache/sysroot/lib -larrow_substrait -larrow_dataset -larrow_acero -larrow_compute -lparquet -larrow -larrow_bundled_dependencies
** libs
using C++ compiler: 'emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.8 (70404efec4458b60b953bc8f1529f2fa112cdfd1)'
using C++20
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c RTasks.cpp -o RTasks.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c altrep.cpp -o altrep.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c array.cpp -o array.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c array_to_vector.cpp -o array_to_vector.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c arraydata.cpp -o arraydata.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c arrowExports.cpp -o arrowExports.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c bridge.cpp -o bridge.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c buffer.cpp -o buffer.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c chunkedarray.cpp -o chunkedarray.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c compression.cpp -o compression.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c compute-exec.cpp -o compute-exec.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c compute.cpp -o compute.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c config.cpp -o config.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c csv.cpp -o csv.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c dataset.cpp -o dataset.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c datatype.cpp -o datatype.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c expression.cpp -o expression.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c extension-impl.cpp -o extension-impl.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c feather.cpp -o feather.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c field.cpp -o field.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c filesystem.cpp -o filesystem.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c io.cpp -o io.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c json.cpp -o json.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c memorypool.cpp -o memorypool.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c message.cpp -o message.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c parquet.cpp -o parquet.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c r_to_arrow.cpp -o r_to_arrow.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c recordbatch.cpp -o recordbatch.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c recordbatchreader.cpp -o recordbatchreader.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c recordbatchwriter.cpp -o recordbatchwriter.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c safe-call-into-r-impl.cpp -o safe-call-into-r-impl.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c scalar.cpp -o scalar.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c schema.cpp -o schema.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c symbols.cpp -o symbols.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c table.cpp -o table.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c threadpool.cpp -o threadpool.o
em++ -std=gnu++20 -DNDEBUG -I/opt/emsdk/upstream/emscripten/cache/sysroot/include -DARROW_ENGINE_STATIC -DARROW_DS_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DPARQUET_STATIC -DARROW_STATIC -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_SUBSTRAIT -DARROW_R_WITH_JSON -I'/root/R/x86_64-pc-linux-gnu-library/4.5/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.5.1/build/include -I/opt/webr/R/build/R-4.5.1/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -fpic -std=gnu++20 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c type_infer.cpp -o type_infer.o
em++ -std=gnu++20 -s SIDE_MODULE=1 -s WASM_BIGINT -s ASSERTIONS=1 -L/opt/webr/wasm/lib -L/opt/webr/wasm/R-4.5.1/lib/R/lib -s USE_BZIP2=1 -s USE_ZLIB=1 -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -Oz -o arrow.so RTasks.o altrep.o array.o array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o expression.o extension-impl.o feather.o field.o filesystem.o io.o json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/opt/emsdk/upstream/emscripten/cache/sysroot/lib -larrow_substrait -larrow_dataset -larrow_acero -larrow_compute -lparquet -larrow -larrow_bundled_dependencies
installing to /tmp/RtmpodX2I9/file16cf61c2f/arrow/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
* creating tarball
packaged installation of 'arrow' as 'arrow_23.0.0_R_x86_64-pc-linux-gnu.tar.gz'
* DONE (arrow)
Packaging: arrow_23.0.0.data
However, the package loading fails at runtime. After library(arrow) within webR I've got, as an R output,
Error in unknown source: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/usr/lib/R/library/arrow/libs/arrow.so':
Could not load dynamic lib: /usr/lib/R/library/arrow/libs/arrow.so
TypeError: resolved is not a function
Is there something that I miss ? Thanks
EDIT :
webr::install("dplyr",repos="https://arnaud-feldmann.github.io/build_rwasm_repo/")
library(dplyr)
starwars %>% select(name, height)
Works, so the repo can actually build packages
But
webr::install("arrow",repos="https://arnaud-feldmann.github.io/build_rwasm_repo/")
library(arrow)
fails with
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/usr/lib/R/library/arrow/libs/arrow.so':
Could not load dynamic lib: /usr/lib/R/library/arrow/libs/arrow.so
TypeError: resolved is not a function
(arrow.so exists)
EDIT2: @lionel- any idea ?
All reactions
Replies: 3 comments 1 reply
Note :
It seems to be related with a symbol evaluation within webR, and the unfound symbol is :
_ZNSt3__26__treeIyNS_4lessIyEENS_9allocatorIyEEE14__assign_multiINS_21__tree_const_iteratorIyPNS_11__tree_nodeIyPvEElEEEEvT_SD_
All reactions
0 replies
Hello. I see you've pinged me but I haven't worked on webR for years 😄.
Maybe @georgestagg or @jeroen have an idea?
All reactions
-
😄 1
0 replies
You can demangle the symbol using c++ filt (prefix with another _):
c++filt __ZNSt3__26__treeIyNS_4lessIyEENS_9allocatorIyEEE14__assign_multiINS_21__tree_const_iteratorIyPNS_11__tree_nodeIyPvEElEEEEvT_SD_
# void std::__2::__tree<unsigned long long, std::__2::less<unsigned long long>, std::__2::allocator<unsigned long long>>::__assign_multi<std::__2::__tree_const_iterator<unsigned long long, std::__2::__tree_node<unsigned long long, void*>*, long>>(std::__2::__tree_const_iterator<unsigned long long, std::__2::__tree_node<unsigned long long, void*>*, long>, std::__2::__tree_const_iterator<unsigned long long, std::__2::__tree_node<unsigned long long, void*>*, long>)
That looks like some libcxx symbol. I suspect some mismatch between c++ versions used in libarrow and the R bindings.
All reactions
1 reply
@jeroen Thanks a lot for your answer.
Strangely I use WebR's official docker image to compile libarrow and the r package, which should have the same libcxx than webR.
I have tried to compile with webr 0.5.6 (and execute with 0.5.6) and to compile with webr 0.5.8 (and to execute with webr 0.5.8). Right now, I have no ideas of what to try. I let it there. If somebody succeeds to build and load arrow within webR, I'm interested.
All reactions
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment