homepage

Hi,
This bug was originally reported at 
https://bugs.launchpad.net/bugs/898172
ctypes/utils.py provides a find_library function which amongst other things will scan the ldconfig -p output on linux to find libraries by name. It applies some logic to filter out incompatible libraries, however the logic is mainly based on uname output which is incorrect.
We noticed because the new Debian/Ubuntu armhf ports have a slightly different ldconfig -p output than the armel ports; one gets ",hard-float" in the output, e.g.:
 ld-linux.so.3 (libc6,hard-float) => /lib/arm-linux-gnueabihf/ld-linux.so.3
there's provision in find_library to allow for certain strings when uname returns certain names:
 mach_map = {
 'x86_64-64': 'libc6,x86-64',
 'ppc64-64': 'libc6,64bit',
 'sparc64-64': 'libc6,64bit',
 's390x-64': 'libc6,64bit',
 'ia64-64': 'libc6,IA-64',
but this is incorrect for multiple reasons:
a) this requires setting utsname properly before running a 32-bits python on a 64-bits kernel (e.g. "linux32 ./foo.py" instead of just "./foo.py"); this shouldn't be needed and breaks 32-bits userspace installations with a 64-bits kernel
b) uname output can be anything really, e.g. i486, i586, i686 etc. on 32-bits x86, or armv5l, armv6l, armv7l etc. on ARM
c) uname output doesn't indicate userspace ABI, a single kernel can support multiple ABIs; for instance ARM kernels can support EABI and OABI (old ABI) syscall ABIs at the same time, and even with the same syscall ABI like EABI the userspace calling conventions might allow for multiple ABIs to be present on the filesystem -- for instance soft-float and hard-float userspace calling conventions
I've attached a patch to ctypes/utils.py in the Launchpad bug which I'll also attach here. It will work for either soft-float or hard-float, but not if "ldconfig -p" lists both types of libraries (as will be the case with biarch or multiarch systems).
It is extremely hard to reproduce correct glibc semantics in find_library, and a linux implementation would necessarily become extremely glibc and linux specific. One possible way is to look at /proc/$pid/maps output to find information about the ABI of the currently running program, and then ask the runtime linker (ld.so) to check whether a given library is compatible or not (--verify). Another way would be to run ldd on sys.executable to find the runtime linker or libc. This is all extremely fragile and linux andglibc specific, and will likely fail in special cases.
Finally, one needs to wonder whether offering "find_library" as an API isn't calling for trouble; dlopen() requires one to state which SOVER should be used, e.g. dlopen("libmagic.so.1"), not dlopen("magic"). Allowing the first SOVER to be used means that the behavior is not determinstic and also means that people wont think of binary compatibility when implementing ctypes-based bindings. I would personally prefer if this API was deprecated and if we recommended for upstreams to use ctypes.cdll.LoadLibrary("libmagic.so.1") constructs.
Cheers,

While I'm at it, find_library also tries creating temp files when running gcc and other issues mention trouble running gcc or propose running ld:
http://bugs.python.org/issue9998
http://bugs.python.org/issue5289
IMHO, calling binutils/gcc is troublesome, it's not necessarily installed on production systems and creating tempfiles when running a program just to locate a library is fragile at best.

This issue is still up to date. Most impacted are distos like for example Buldroot (http://buildroot.org/). Usual production rootfs provide neither gcc nor /sbin/ldconfig nor objdump. So find_library() is doomed from the very beginning.
Such packages like pyusb, pyudev etc. rely on find_library() and are not usable in such environment. Pyusb developers had to create an extra backend to overcome this issue (https://github.com/walac/pyusb/pull/29).
So common solution would be very useful for such environments.

I am still encountering this issue when using pyudev on Python 3.8.5.