-
Notifications
You must be signed in to change notification settings - Fork 455
Can't Link Custom Static Libraries #1766
-
So the project I am trying to cross-compile uses a client socket to communicate with another program on my Raspberry Pi. The Rust socket functions didn't work as well on the Pi as they did on my desktop, so I'm going to use a C++ socket instead. I compiled that into a .a static library, saved it in ./lib, tested with cargo run, it worked perfectly. When I went to cross-compile it for the Pi, I recompiled the static library for aarch64 successfully but when I tried cross build, I got this:
cross build -vv --target=aarch64-unknown-linux-gnu
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup --verbose toolchain add stable-x86_64-unknown-linux-gnu --profile minimal
debug: read metadata version: 12
debug: updating existing install for 'stable-x86_64-unknown-linux-gnu'
debug: toolchain directory: /home/aidan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu
info: syncing channel updates for stable-x86_64-unknown-linux-gnu
debug: creating temp file path=/home/aidan/.rustup/tmp/2lnmyafgnvw_rhy5_file
debug: downloading file url=https://static.rust-lang.org/dist/channel-rust-stable.toml.sha256
debug: downloading with reqwest
debug: deleted temp file path=/home/aidan/.rustup/tmp/2lnmyafgnvw_rhy5_file
debug: toolchain is already up to date
stable-x86_64-unknown-linux-gnu unchanged - rustc 1.94.1 (e408947bf 2026年03月25日)
info: checking for self-update (current version: 1.29.0)
debug: downloading file url=https://static.rust-lang.org/rustup/release-stable.toml
debug: downloading with reqwest
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker
+ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig' -e 'PKG_CONFIG_SYSROOT_DIR=/' -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=aidan' --rm --user 1000:1000 -v /home/aidan/.xargo:/xargo:z -v /home/aidan/.cargo:/cargo:z -v /cargo/bin -v '/media/aidan/Local Disc B/Documents/Civic Mods/AidF Infotainment System/Nav System/Tests/Rust Socket Client Test:/project:z' -v /home/aidan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v '/media/aidan/Local Disc B/Documents/Civic Mods/AidF Infotainment System/Nav System/Tests/Rust Socket Client Test/target:/target:z' -w /project -i -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build -vv --target=aarch64-unknown-linux-gnu'
Dirty LibAIBus_Test v0.1.0 (/project): the precalculated components changed
Compiling LibAIBus_Test v0.1.0 (/project)
Running `CARGO=/rust/bin/cargo CARGO_CFG_DEBUG_ASSERTIONS='' CARGO_CFG_FEATURE='' CARGO_CFG_PANIC=unwind CARGO_CFG_TARGET_ABI='' CARGO_CFG_TARGET_ARCH=aarch64 CARGO_CFG_TARGET_ENDIAN=little CARGO_CFG_TARGET_ENV=gnu CARGO_CFG_TARGET_FAMILY=unix CARGO_CFG_TARGET_FEATURE=neon CARGO_CFG_TARGET_HAS_ATOMIC=128,16,32,64,8,ptr CARGO_CFG_TARGET_OS=linux CARGO_CFG_TARGET_POINTER_WIDTH=64 CARGO_CFG_TARGET_VENDOR=unknown CARGO_CFG_UNIX='' CARGO_ENCODED_RUSTFLAGS='' CARGO_MANIFEST_DIR=/project CARGO_MANIFEST_PATH=/project/Cargo.toml CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=LibAIBus_Test CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' DEBUG=true HOST=x86_64-unknown-linux-gnu LD_LIBRARY_PATH='/target/debug:/target/debug/deps:/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib' NUM_JOBS=16 OPT_LEVEL=0 OUT_DIR=/target/aarch64-unknown-linux-gnu/debug/build/LibAIBus_Test-b344bd3b621a15c9/out PROFILE=debug RUSTC=rustc RUSTC_LINKER=aarch64-linux-gnu-gcc RUSTDOC=rustdoc TARGET=aarch64-unknown-linux-gnu /target/debug/build/LibAIBus_Test-b46b787b997a2534/build-script-build`
[LibAIBus_Test 0.1.0] cargo:rustc-link-lib=dylib=stdc++
[LibAIBus_Test 0.1.0] cargo:rustc-link-search=/lib
[LibAIBus_Test 0.1.0] cargo:rustc-link-lib=static=aibusclient
Running `CARGO=/rust/bin/cargo CARGO_BIN_NAME=LibAIBus_Test CARGO_CRATE_NAME=LibAIBus_Test CARGO_MANIFEST_DIR=/project CARGO_MANIFEST_PATH=/project/Cargo.toml CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=LibAIBus_Test CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH=/target/debug/deps OUT_DIR=/target/aarch64-unknown-linux-gnu/debug/build/LibAIBus_Test-b344bd3b621a15c9/out rustc --crate-name LibAIBus_Test --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=218 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=a256ee47af2eaf3a -C extra-filename=-5f106ec52f21f0c5 --out-dir /target/aarch64-unknown-linux-gnu/debug/deps --target aarch64-unknown-linux-gnu -C linker=aarch64-linux-gnu-gcc -C incremental=/target/aarch64-unknown-linux-gnu/debug/incremental -L dependency=/target/aarch64-unknown-linux-gnu/debug/deps -L dependency=/target/debug/deps -L /lib -l dylib=stdc++ -l static=aibusclient`
error: linking with `aarch64-linux-gnu-gcc` failed: exit status: 1
|
= note: "aarch64-linux-gnu-gcc" "/target/aarch64-unknown-linux-gnu/debug/deps<sysroot>c81BRu5/symbols.o" "<15 object files omitted>" "-Wl,--as-needed" "-Wl,-Bdynamic" "-lstdc++" "-Wl,-Bstatic" "-laibusclient" "<sysroot>/lib/rustlib/aarch64-unknown-linux-gnu/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,libcfg_if-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,liblibc-*,librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/target/aarch64-unknown-linux-gnu/debug/deps<sysroot>c81BRu5/raw-dylibs" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/lib" "-L" "<sysroot>/lib<sysroot>lib/aarch64-unknown-linux-gnu/lib" "-o" "/target/aarch64-unknown-linux-gnu/debug/deps/LibAIBus_Test-5f106ec52f21f0c5" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
= note: some arguments are omitted. use `--verbose` to show all linker arguments
= note: /usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: cannot find -laibusclient
collect2: error: ld returned 1 exit status
error: could not compile `LibAIBus_Test` (bin "LibAIBus_Test") due to 1 previous error
Caused by:
process didn't exit successfully: `CARGO=/rust/bin/cargo CARGO_BIN_NAME=LibAIBus_Test CARGO_CRATE_NAME=LibAIBus_Test CARGO_MANIFEST_DIR=/project CARGO_MANIFEST_PATH=/project/Cargo.toml CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=LibAIBus_Test CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH=/target/debug/deps OUT_DIR=/target/aarch64-unknown-linux-gnu/debug/build/LibAIBus_Test-b344bd3b621a15c9/out rustc --crate-name LibAIBus_Test --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=218 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=a256ee47af2eaf3a -C extra-filename=-5f106ec52f21f0c5 --out-dir /target/aarch64-unknown-linux-gnu/debug/deps --target aarch64-unknown-linux-gnu -C linker=aarch64-linux-gnu-gcc -C incremental=/target/aarch64-unknown-linux-gnu/debug/incremental -L dependency=/target/aarch64-unknown-linux-gnu/debug/deps -L dependency=/target/debug/deps -L /lib -l dylib=stdc++ -l static=aibusclient` (exit status: 1)
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
The static library was recompiled for aarch64, it's in the same lib folder, the title should be correct (libaibusclient.a), why doesn't cross seem to be seeing it?
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
That's the issue - the symlink. When cross mounts your project into Docker at /project, it copies the symlink itself but the target it points to doesn't exist inside the container. So the linker sees a broken symlink and can't find the .a file.
The fix: replace the symlink with the actual file. Either copy it directly:
cp -L lib/libaibusclient.a lib/libaibusclient.a
Or if you want to keep building the lib separately and having it auto-update, add a pre-build step to your workflow that dereferences the symlink before running cross:
/project, any link paths need to be relative to that mount point to work inside the container.
The most reliable fix is using a build.rs script that leverages CARGO_MANIFEST_DIR. This environment variable points to the crate root both on your host and inside the cross environment.
// build.rs fn main() { let dir = std::env::var("CARGO_MANIFEST_DIR").unwrap(); println!("cargo:rustc-link-search=native={}/lib", dir); println!("cargo:rustc-link-lib=static=your_lib_name"); // You likely need to link the C++ runtime explicitly as well println!("cargo:rustc-link-lib=stdc++"); }
One gotcha to watch out for: ensure your static library follows the libname.a naming convention. If it's just name.a, the linker will usually skip it. IIRC, some older cross images might also have glibc version mismatches if your .a was built on a much newer toolchain than what's in the 0.2.5 image, so keep an eye on that if you get "undefined reference" errors. Lmk if that helps.
Beta Was this translation helpful? Give feedback.
All reactions
-
Unfortunately that yielded the same error. Tried moving the c++ link towards the end, same issue.
The naming convention is correct.
aarch64-linux-gnu-ar --version
GNU ar (GNU Binutils for Ubuntu) 2.42
Copyright (C) 2024 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.
aarch64-linux-gnu-g++ --version
aarch64-linux-gnu-g++ (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Beta Was this translation helpful? Give feedback.
All reactions
-
Hmm, that's surprising - I tested the CARGO_MANIFEST_DIR approach and it compiled successfully on my end for a similar setup (aarch64 static lib linked via build.rs).
A couple things to check:
First, make sure you ran cross clean before rebuilding. Cross caches build script output pretty aggressively, so if the old /lib path is still cached, it'll keep using that even after you change build.rs.
Second, can you add some debug output to your build.rs to verify what paths are actually being used? Something like:
fn main() { let dir = std::env::var("CARGO_MANIFEST_DIR").unwrap(); eprintln!("=== MANIFEST_DIR: {} ===", dir); let lib_path = format!("{}/lib", dir); eprintln!("=== LIB EXISTS: {} ===", std::path::Path::new(&format!("{}/libaibusclient.a", lib_path)).exists()); println!("cargo:rustc-link-search=native={}", lib_path); println!("cargo:rustc-link-lib=static=aibusclient"); println!("cargo:rustc-link-lib=dylib=stdc++"); }
Run cross build -vv --target=aarch64-unknown-linux-gnu 2>&1 | grep "===" and share what it prints. That'll tell us whether CARGO_MANIFEST_DIR resolves to /project inside the container and whether it can actually see the .a file at that path.
Also worth checking - do you have cargo:rustc-link-search=/lib anywhere else? Like in a .cargo/config.toml or a [package.links] section in Cargo.toml?
Beta Was this translation helpful? Give feedback.
All reactions
-
It resolved to /project but can't find the library... hmm....
cross build -vv --target=aarch64-unknown-linux-gnu 2>&1 | grep "==="
[LibAIBus_Test 0.1.0] === MANIFEST_DIR: /project ===
[LibAIBus_Test 0.1.0] === LIB EXISTS: false ===
Nothing like that in Cargo.toml, here is my Cross.toml:
[target.aarch64-unknown-linux-gnu.env]
passthrough = [
"PKG_CONFIG_ALLOW_CROSS=1",
"PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig",
"PKG_CONFIG_SYSROOT_DIR=/"
]
The .a file is a symbolic link so I don't have to keep copying it, could that have something to do with it?
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
That's the issue - the symlink. When cross mounts your project into Docker at /project, it copies the symlink itself but the target it points to doesn't exist inside the container. So the linker sees a broken symlink and can't find the .a file.
The fix: replace the symlink with the actual file. Either copy it directly:
cp -L lib/libaibusclient.a lib/libaibusclient.a
Or if you want to keep building the lib separately and having it auto-update, add a pre-build step to your workflow that dereferences the symlink before running cross:
I reproduced this exact error in a sandbox - symlink works fine with cargo build locally but breaks inside the Docker container because the target path doesn't exist there.
Beta Was this translation helpful? Give feedback.
All reactions
-
Thanks! Looks like my C++ library is out of date, I'm now getting an "undefined reference to `std::__throw_bad_array_new_length()." Hopefully that is an easier fix.
Beta Was this translation helpful? Give feedback.
All reactions
-
😄 1
-
Yeah that's a libstdc++ version mismatch. Your host compiles the C++ lib with GCC 13.3 (Ubuntu 24.04), but the cross container 0.2.5 ships with GCC 5. The std::__throw_bad_array_new_length symbol was added in GCC 12's libstdc++, so the old container can't resolve it.
Easiest fix: build the C++ lib inside the cross container using a pre-build hook in Cross.toml. That way it uses the same toolchain the linker expects:
[target.aarch64-unknown-linux-gnu] pre-build = [ "apt-get update && apt-get install -y g++", "g++ -c /project/lib/client.cpp -o /project/lib/client.o", "ar rcs /project/lib/libaibusclient.a /project/lib/client.o" ]
This compiles the .a inside the container with the container's GCC, so the symbols match. You'd still keep your build.rs with the CARGO_MANIFEST_DIR path.
Alternative: use a newer cross image that has a modern GCC. You can set a custom image in Cross.toml:
[target.aarch64-unknown-linux-gnu] image = "ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main"
The main tag usually tracks the latest and has a newer toolchain than 0.2.5.
Beta Was this translation helpful? Give feedback.
All reactions
-
Thanks!! I will have to de-reference a lot...
With the number of components in this project, it makes more sense to reuse some code.
Beta Was this translation helpful? Give feedback.
All reactions
-
😄 1
-
I tried this during my lunch break and had one more quick question- g++ can't find the /project directory, I ran an ls -l and it didn't even look like it was mounted. Is there a way to mount the working directory in pre-build?
Beta Was this translation helpful? Give feedback.