Getting "Not found" message when running a 32-bit binary on a 64-bit system

Question

I have currently a strange problem on debian (wheezy/amd64).

I have created a chroot to install a server (i can't give any more detail about it, sorry). Let's call its path /chr_path/. To make things easy, I have initialized this chroot with a debootstrap (also wheezy/amd64).

All seemed to work well inside the chroot but when I started the installer script of my server I got : zsh: Not found /some_path/perl (the installer includes a perl binary for some reasons)

Naturally, I checked the /some_path/ location and I found the "perl" binary. file in chroot environment returns :

/some_path/perl ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.2.5, not stripped

The file exists, seems ok, has correct rights. I can use file, ls, vim on it but as soon as I try to execute it - ./perl for example - I get : zsh: Not found ./perl.

This situation is quite understandable for me. Moreover :

• I can execute other basic binaries (/bin/ls,...) in the chroot without getting errors
• I have the same problems for other binaries that came with the project
• When I try to execute the binary from the main root (/chr_path/some_path/perl), it works.
• I have tried to put one of the binaries with a copy of my ls. I checked that the access rights were the same but this didn't change anything (one was working, and the other wasn't)

• Gilles 'SO- stop being evil': This is the same issue as "No such file or directory" lies on Optware installed binaries. Note that your Perl is a 32-bit executable. You're missing the 32-bit runtime system (libc6-i386 package, or ia32-libs if you want a lot of libraries).
• Elenaher: @Gilles : Thanks a lot ! An aptitude install ia32-libs has solved the problem !! I had seen that the perl was 32 bits but since it was working on the main system (same distrib) I had just assumed it wasn't linked. Actually, I must have installed the 32-bit runtime system on the main system at some point.
• Caleb: @Gilles: I think I would add that as a concise answer instead of marking this as a duplicate question. The environment is different enough that even though the problem is the same, searching people are more likely to hit one or the other.
• Michael Mrozek: @Caleb We don't delete duplicates for exactly that reason; searchers that find this one will just follow the duplicate link to the other post. If this is the same problem it should probably just be closed
• Gilles 'SO- stop being evil': @MichaelMrozek I've changed my mind on this question: while the underlying issue is the same, the concrete remedy is a bit different (not mixing ARM ABIs in one case, enabling 32-bit support on an amd64 Linux distribution in the other). So I guess this question belongs open after all.

Answer 1

When you fail to execute a file that depends on a "loader", the error you get may refer to the loader rather than the file you're executing.

• The loader of a dynamically-linked native executable is the part of the system that's responsible for loading dynamic libraries. It's something like /lib/ld.so or /lib/ld-linux.so.2, and should be an executable file.
• The loader of a script is the program mentioned on the shebang line, e.g. /bin/sh for a script that begins with #!/bin/sh. (Bash and zsh give a message "bad interpreter" instead of "command not found" in this case.)

The error message is rather misleading in not indicating that the loader is the problem. Unfortunately, fixing this would be hard because the kernel interface only has room for reporting a numeric error code, not for also indicating that the error in fact concerns a different file. Some shells do the work themselves for scripts (reading the #! line on the script and re-working out the error condition), but none that I've seen attempt to do the same for native binaries.

ldd won't work on the binaries either because it works by setting some special environment variables and then running the program, letting the loader do the work. strace wouldn't provide any meaningful information either, since it wouldn't report more than what the kernel reports, and as we've seen the kernel can't report everything it knows.

This situation often arises when you try to run a binary for the right system (or family of systems) and superarchitecture but the wrong subarchitecture. Here you have ELF binaries on a system that expects ELF binaries, so the kernel loads them just fine. They are i386 binaries running on an x86_64 processor, so the instructions make sense and get the program to the point where it can look for its loader. But the program is a 32-bit program (as the file output indicates), looking for the 32-bit loader /lib/ld-linux.so.2, and you've presumably only installed the 64-bit loader /lib64/ld-linux-x86-64.so.2 in the chroot.

You need to install the 32-bit runtime system in the chroot: the loader, and all the libraries the programs need. From Debian wheezy onwards, if you want both i386 and x86_64 support, start with an amd64 installation and activate multiarch support: run dpkg --add-architecture i386 then apt-get update and apt-get install libc6:i386 zlib1g:i386 ... (if you want to generate a list of the dependencies of Debian's perl package, to see what libraries are likely to be needed, you can use aptitude search -F %p '~Rdepends:^perl$ ~ri386'). You can pull in a collection of common libraries by installing the ia32-libs package (you need to enable multiarch support first). On Debian amd64 up to wheezy, the 32-bit loader is in the libc6-i386 package. You can install a bigger set of 32-bit libraries by installing ia32-libs.

• Nathan Osman: Is this the only thing that can trigger the error message? I've got the 32-bit libraries installed and here's the output of ldd but I still get the same error.
• Gilles 'SO- stop being evil': @NathanOsman Probably unix.stackexchange.com/questions/76490/…
• Nathan Osman: I tried installing lsb-core but that didn't seem to help. I think I better open a new question for this.
• Aster: Thank you for this, you just ended two days of head-scratching. I thought everything was being statically compiled but it was not!
• einpoklum: Sometimes, the architecture code is 686 or i686 rather than 386 or i386, and it's still 32-bits.

Answer 2

Run ldd(1) on your perl binary. Often the seemingly confusing Not found error on a file that is clearly there is because one of the shared libraries used by the program is not found.

So it is possible that your chroot is incomplete with respect to the shared libraries needed by your binaries.

• Elenaher: Actually I get : perl is not a dynamic executable when I am in the chroot and I get the correct list of dependencies from outside. I am currently checking if there's something strange but I used a debootstrap to avoid this kinds of lack and have many libs already in place (there's a perl executable in the chroot system that runs well but it's a different version ; maybe I will just do some symbolic link ?)
• camh: To be honest, I would have expected debootstrap to have produced a complete chroot, so I would not expect my answer to be correct in that respect. But I've run across the missing library in chroot problem before so I thought I'd see if my answer would fly.
• Elenaher: cf. comment to Gilles on main post : You were right. There was some libs missing. The main advantage of debootstrap is that I could solve the problem with a basic aptitude install :)