Search code examples
yocto

Yocto Build Maxium PARALLEL_MAKE, BB_NUMBER_THREADS

I recently tried to start the yocto build task with leaving PARALLEL_MAKE and BB_NUMBER_THREADS blank, e.g. commented out and starting the Build on a Host with 48 or 64 vCPU's. The build was failing. With 32 vCPU's it still works. Yocto is still old Krogoth and cannot be updated currently. Does anybody knows if there is a limitation going further than 32 vCPU's? Is this just for older Yocto versions or generally an Issue?

Update: The error is related to a custom recipe for compiling a rnnoise-plugin:

../rnnoise-plugin/1.0-r0/temp/log.do_compile.1449627

rnnoise-plugin/1.0-r0/asound-pcm-plugin-rnnoise.c:16:28: fatal error: alsa/asoundlib.h: No such file or directory
#include <alsa/asoundlib.h>
                            ^

As mentioned above, i only get this error when compiling on a 48 or 64 vCPU Host with PARALLEL_MAKE and BB_NUMBER_THREADS left blank. If i set PARALLEL_MAKE="-j 32" and BB_NUMBER_THREADS="32" it compiles without any error.

Solution

  • It seems you run into race conditions when compiling with that many threads.

    The yocto reference recommends setting PARALLEL_MAKE so make uses a maximum of 20 threads when using more than one physical CPU. If your system only has a single CPU this might be a virtualization problem.

    For single socket systems (i.e. one CPU), you should not have to override this variable to gain optimal parallelism during builds. However, if you have very large systems that employ multiple physical CPUs, you might want to make sure the PARALLEL_MAKE variable is not set higher than "-j 20".

    Yocto Project - PARALLEL_MAKE

    For increased building speed you can try to fix this dependency issue by patching the appropriate Makefile (Debugging Parallel Make Races)

    Edit:

    It is possible to set PARALLEL_MAKE for a particular package when overwriting in its recipe which allows to compile one particular package with make -j 2 e.g. while other packages use maximum parallelism.

    So it might be a suitable workaround to set PARALLEL_MAKE to a lower value in the recipe of the package where the dependency issue occurs.

    But keep in mind that the yocto reference also recommends to limit BB_NUMBER_THREADS (no package-specific parameter) to 20 on large systems with multiple CPU's which might be the case when using hardware virtualization.