I am setting up a foreign chroot environment to build for architectures other than amd64 from a GitLab CI image. Steps were mostly taken from https://www.hellion.org.uk/blog/posts/foreign-chroots-with-schroot-and-qemu/, except that I am skipping the schroot/sbuild part.
- export CROSS_ARCH=armhf
- export CROSS_ROOT=/opt/chroot/$CROSS_ARCH
- export DISTRO=stretch
- export CROSS_MIRROR=http://deb.debian.org/debian/
- apt-get update
- apt-get -y install debootstrap qemu-user-static binfmt-support
- mkdir -p $CROSS_ROOT
- debootstrap --variant=buildd --include=fakeroot,build-essential --arch=$CROSS_ARCH --foreign $DISTRO $CROSS_ROOT $CROSS_MIRROR
- mkdir -p $CROSS_ROOT/usr/bin
- cp /usr/bin/qemu-arm-static $CROSS_ROOT/usr/bin/
- chroot $CROSS_ROOT ./debootstrap/debootstrap --second-stage
When I now try to run a command in the target environment like this:
chroot $CROSS_ROOT qemu-arm-static uname -a
the command exits with an error (nonzero exit status), but no error message is printed. It works, however, if I specify the path:
chroot $CROSS_ROOT qemu-arm-static /bin/uname -a
And it gives me the following output, which indicates I am running inside the armhf environment:
Linux runner--azerasq-project-40807358-concurrent-0 5.4.109+ #1 SMP Wed Jun 16 20:00:10 PDT 2021 armv7l GNU/Linux
Oddly, the following works:
chroot $CROSS_ROOT qemu-arm-static /bin/bash -c "uname -a"
i.e. full path to bash, but no path for the command after -c.
Suspecting that there could be something wrong with $PATH, I ran:
chroot $CROSS_ROOT qemu-arm-static /bin/bash -c set
I get all of the GitLab-specific variables, as well as a bunch of others, including the following ones:
MACHTYPE=arm-unknown-linux-gnueabihf
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
When I run
chroot $CROSS_ROOT qemu-arm-static /usr/bin/env
some variables (such as $MACHTYPE) are missing but $PATH is the same. So $PATH seems to be set correctly, and a diff of the outputs (after sorting) does not indicate anything that looks related – the extra variables for bash -c set look mostly bash-specific.
Why won’t qemu-arm-static accept binaries without a full path if they are on $PATH? Where else should I look to debug?
Why won’t qemu-arm-static accept binaries without a full path if they are on $PATH?
Because qemu-user is not a shell, it doesn't have code that would search the PATH. This is the piece of qemu-user code that opens executable image when it's started as in the examples that you give, and as you can see here exec_path comes directly from the command line.
On the other hand you can install qemu-user as a binfmt-misc handler, in which case the shell will do the PATH search and the kernel will invoke qemu-user with an open file descriptor of the executable file in the AT_EXECFD entry in the aux vectors.