I am seeing something really weird inside a chroot-ed Debian armel environment.
But first, a bit of backstory... This is long, but the question is complex and any potential help depends on knowing the full story.
I have an embedded ARM SoC that runs Linux - more specifically,
a Debian armel Lenny on a 2.6.17 kernel. The Debian distro itself is
easily upgradeable to later versions (sudo apt-get dist-upgrade)
and can thus be brought up to speed, to the armel versions of
squeeze or even wheezy.
The problem is that the kernel is a custom one... The ARM SoC in question is not part of the mainline kernel, so it is pretty much abandoned at 2.6.17.
If you know how Linux and GLIBC work, you can already see the problem - GLIBC versions are compiled with a minimum supported kernel version... Which has moved way past 2.6.17. So if we try to e.g. chroot to a Debian squeeze...
$ # From inside the little ARM machine running Debian Lenny
$ sudo debootstrap --arch armel squeeze /squeeze \
http://ftp.whateverCountry.debian.org/debian
$ sudo -i
# mount -t proc none /squeeze/proc
# mount -t sysfs none /squeeze/sys
# mount -t devpts none /squeeze/dev/pts
# chroot /squeeze
Fatal: Kernel too old
...we see a message from the GLIBC of squeeze, telling us that
it was not compiled to work with this old kernel (2.6.17).
The same problem happens with wheezy, too - since it is newer than squeeze - and will in fact happen with any Debian version from now on, since their GLIBC won't work on my 2.6.17 kernel.
At first I thought this was a deal breaker - but then I realized that I can in theory recompile GLIBC to work with the older kernel my SoC is using... But I'd need an identical environment to what was used to build the libc6 package in e.g. Debian squeeze.
I am guessing the compilation of GLIBC and the preparation of the libc6_2.11.3-4.deb file is done via automated cross-compiling machinery invented by the Gods of Debian.
I am no God... nor could I find anything in Google about how
to become one - i.e. how to use my Core i5 as a host, to
cross-compile GLIBC using the exact same settings that the packaged
version (inside Debian squeeze) is using.
So I tricked it - I figured out how to setup the ARM version
of Debian squeeze on my Core i5 (a technique that uses a static
version of the qemu-arm binary).
Once I chrooted in my x86-hosted version of Debian-armel-squeeze,
I was able to simply...
$ cd /var/tmp
$ apt-get source libc6
...
$ # edit this in - compile for my kernel...
$ vi eglibc-2.11.3/debian/sysdeps/linux.mk
...
MIN_KERNEL_SUPPORTED := 2.6.17
...
$ export DEB_BUILD_OPTS="nocheck parallel=1"
$ cd eglibc-2.11.3
$ dpkg-buildpackage -b -d -us -uc
...and after 3 hours (the Core i5-hosted chrooted version of
Debian-armel-squeeze is much slower than a native machine...)
I got my libc6 .deb package. It would probably take 3 months
to do this build in my SoC, so I am not complaining.
Going back inside my real ARM SoC, I copied all the libc files (.so) of the new package over the default ones of squeeze and tried to chroot...
# chroot squeeze/
root@ttsiodras:/#
Yes! It worked! (or so it seemed)
My custom libc reported from inside the chroot:
# /lib/libc.so.6
GNU C Library (Debian EGLIBC 2.11.3-4) stable release version 2.11.3, by Roland McGrath et al.
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.4.5.
Compiled on a Linux 2.6.26 system on 2014-10-23.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
Support for some architectures added on, not maintained in glibc core.
BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.
Things seemed to work - I copied a file, invoked ls...
But when I tried to use apt-get to install some apps from squeeze, I started getting... some unexpected errors:
# apt-get install indent
Reading package lists... Done
Building dependency tree... Done
The following NEW packages will be installed:
indent
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 110 kB of archives.
After this operation, 516 kB of additional disk space will be used.
Get:1 http://ftp.gr.debian.org/debian/ squeeze/main indent armel 2.2.11-1 [110 kB]
Fetched 110 kB in 0s (236 kB/s)
tar: ./control: Cannot utime: Function not implemented
tar: ./md5sums: Cannot utime: Function not implemented
tar: .: Cannot utime: Function not implemented
tar: Exiting with failure status due to previous errors
dpkg-deb: subprocess tar returned error exit status 2
dpkg: error processing /var/cache/apt/archives/indent_2.2.11-1_armel.deb (--unpack):
subprocess dpkg-deb --control returned error exit status 2
configured to not write apport reports
rm: cannot remove `/var/lib/dpkg/tmp.ci': Function not implemented
dpkg: error while cleaning up:
subprocess rm cleanup returned error exit status 1
Errors were encountered while processing:
/var/cache/apt/archives/indent_2.2.11-1_armel.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
Oh-oh... a bunch of Function not implemented. That sounds like GLIBC reporting
that basic things are not working...
I managed to strace (don't ask how) and figured out that all the -at functions
are failing: openat, mkdirat, renameat, etc - they are all reporting ENOSYS.
It appears that I was only partially successful - some system calls are failing in my new GLIBC.
Is it impossible to compile a squeeze or wheeze GLIBC to execute under 2.6.17?
Any ideas/pointers on what I did wrong and/or how to proceed would be much appreciated...
I made it :-)
I basically followed Gilles's advice (from UNIX stackexchange) and decided to do it properly: i.e. manage a complete cross-compilation of GLIBC. I started from crosstool-ng, and was initially disappointed - seeing that it didn't support my old kernel. I kept at it, though - manually editing the configuration file saved by crosstool-ng to do changes like these on the default arm-gnueabi build configuration:
$ ct-ng arm-unknown-linux-gnueabi
$ ct-ng menuconfig
...
$ vi .config
$ cat .config
...
CT_KERNEL_VERSION="2.6.17"
CT_KERNEL_V_2_6_17=y
CT_LIBC_VERSION="2.13"
CT_LIBC_GLIBC_V_2_13=y
CT_LIBC_GLIBC_MIN_KERNEL_VERSION="2.6.9"
CT_LIBC_GLIBC_MIN_KERNEL="2.6.9
...
$ ct-ng +libc
After numerous tests and failed attempts, the above changes did it - I got a compiled version of GLIBC that would work with my kernel, and copied the resulting files to my Debian Lenny ARM machine:
$ cd .build/arm-unknown-linux-gnueabi/build/build-libc-final/
$ tar zcpf newlibc.tgz $(find . -type f -iname \*.so)
$ scp newlibc.tgz root@mybook:.
I went all the way and moved past squeeze: I debootstrapped a /wheezy and then - very carefully - overwrote the GLIBC versions of the armel-debootstrapped /wheezy with my own:
# # In the ARM machine
# cd /wheezy/lib/arm-linux-gnueabi/
# mv /var/tmp/ohMyGod/libc.so libc-2.13.so
# mv /var/tmp/ohMyGod/rt/librt.so librt-2.13.so
...
...etc, making sure I didn't miss any shared libraries.
Finally, I copied over the ldd and ldconfig binaries (which were also part of GLIBC), and chrooted inside my /wheezy.
It worked.
I can only assume that the compilation of GLIBC from a chroot-ed qemu-arm somehow messed things up - maybe the configure process detects some stuff from the running environment - whereas the cross-compilation can't be misled.
So naturally I moved to the next step, and used a busybox-static shell to replace the {/bin,/sbin,...} folders of my old lenny with the wheezy ones - and rebooted into my brand new Wheezy :-)
I hereby claim that my WD MyBook World Edition is the only one on the planet running Debian Wheezy :-) If anyone else is interested, I can upload a tarball of the libc files someplace.