We are building our Yocto (2.2) Project through AWS CodeBuild (Ubuntu 16.04) and are trying to configure it with Amazon EFS so that our build/tmp, build/sstate-cache, and build/cache directories are persisted between subsequent builds.
Our setup with CodeBuild and EFS is that we clone our repo into the EFS and run our Bitbake build from there.
Currently without Amazon EFS, our builds run with no issues, but once the EFS is enabled and running from that file system, our builds begin to fail with errors like:
Exception: OSError: [Errno 31] Too many links './usr/lib/locale/aa_DJ/LC_MEASUREMENT' -> '/codebuild/readonly/firmware_repo/repo/build/tmp/work/cortexa7hf-neon-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/glibc-binary-localedata-aa-dj/./usr/lib/locale/aa_DJ/LC_MEASUREMENT'
cp: cannot create hard link '/efs/firmware_repo/repo/build/tmp/deploy/licenses/glibc-locale/generic_GPLv2' to '/efs/firmware_repo/repo/build/tmp/work/cortexa7hf-neon-poky-linux-gnueabi/glibc-locale/2.24-r0/license-destdir/glibc-locale/generic_GPLv2': Too many links
We see this error happening in bc_1.06.bb:do_populate_lic, netbase_5.3.bb:do_populate_lic, glibc-locale_2.24.bb:do_populate_lic
I have looked into this issue and it looks to be because Amazon EFS has a cap on the number of hard links that a file can have 1. Are there any ways to either increase the cap on the number of hard links a file can have in Amazon EFS or reduce the number of links a file has during the Bitbake build process?
Some resources that I've used to look into this problem:
OE/Yocto Project uses hardlinks heavily in TMPDIR to construct sysroots quickly and efficently so that won't work well with limits on the numbers.
TMPDIR is designed to be semi-disposable. Sharing only sstate-cache between builds should get a pretty similar speed improvement and that doesn't contain hardlinks so have you tried just persisting that? That should give reasonable performance.