I have been trying to get my CC toolchain on an EFS mount so my jenkins slaves can use them. What I have noticed is no matter how I put the toolchain on the EFS (cp directly, untar directly) it always loses its component paths eg.
/efs/mount/path/arm-linux-gnueabihf-gcc -print-prog-name=cc1
returns
cc1
instead of
/efs/mount/path/cc1path/cc1
If I untar the file on ebs and do the same command I get
/home/dir/arm-linux-gnueabihf-gcc -print-prog-name=cc1
returns
/home/dir/cc1path/cc1
I cannot find a way to preserve these paths, what is going on with EFS? How can I achieve my goal of preserving the paths on EFS?
UPDATE:
after diving in further running strace yeilds: strace /efs/mount/path/arm-linux-gnueabihf-gcc -print-prog-name=cc1
stat64("/mnt/crosscompilers/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../libexec/gcc/ld", 0xffc0d148) = -1 ENOENT (No such file or directory)
stat64("/mnt/crosscompilers/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../libexec/gcc/arm-linux-gnueabihf/ld", 0xffc0d148) = -1 ENOENT (No such file or directory)
stat64("/mnt/crosscompilers/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/arm-linux-gnueabihf/4.7.3/ld", 0xffc0d148) = -1 ENOENT (No such file or directory)
stat64("/mnt/crosscompilers/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/ld", {st_mode=S_IFREG|0755, st_size=1017424, ...}) = 0
stat64("/mnt/crosscompilers/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/arm-linux-gnueabihf/ld", 0xffc0d148) = -1 ENOENT (No such file or directory)
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
write(1, "ld\n", 3ld
) = 3
exit_group(0) = ?
+++ exited with 0 +++
while the working directory yeilds this:
stat64("/home/ubuntu/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../libexec/gcc/ld", 0xffbb1de8) = -1 ENOENT (No such file or directory)
stat64("/home/ubuntu/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../libexec/gcc/arm-linux-gnueabihf/ld", 0xffbb1de8) = -1 ENOENT (No such file or directory)
stat64("/home/ubuntu/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/arm-linux-gnueabihf/4.7.3/ld", 0xffbb1de8) = -1 ENOENT (No such file or directory)
stat64("/home/ubuntu/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/ld", {st_mode=S_IFREG|0755, st_size=1017424, ...}) = 0
access("/home/ubuntu/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/ld", X_OK) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
write(1, "/home/ubuntu/altera-linux/linaro"..., 171/home/ubuntu/altera-linux/linaro/gcc-linaro-arm-linux-gnueabihf-4.7-2012.11-20121123_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.7.3/../../../../arm-linux-gnueabihf/bin/ld
) = 171
exit_group(0) = ?
+++ exited with 0 +++
Take note of the 'access' line in the working output. The actual function print_prog_name runs is here. it says
if (stat (temp, &st) >= 0
&& ! S_ISDIR (st.st_mode)
&& access (temp, mode) == 0)
return temp;
but as you can tell from the output of stat, the file is not a directory. And both files do exist.
TL;DR;
EFS uses 64-bit inodes which 32-bit binaries cannot understand without help from kernel boot params. in our case update /etc/default/grub with GRUB_CMDLINE_LINUX="nfs.enable_ino64=0".
ANSWER;
After some digging we found out it was due to EFS returning a very large inode that our 32 bit binaries couldn't read properly. the kernel can intercept nfs requests and adjust them so 32 bit binaries can understand 64 bit inodes by setting GRUB_CMDLINE_LINUX="nfs.enable_ino64=0" in /etc/default/grub and rebooting.
It was very interesting this was the only error we saw, and some of our binaries still function properly even though they were 32 bit. We never figured out the reason for that.