Search code examples
numpycompilationfacebook-infer

How can I run Facebook infer on numpy library?

I want to run Infer on numpy library to detect its weaknesses. When I run the following command:

infer run -- gcc -c /numpy/numpy/numpy/core/src/multiarrayabstractdtypes.c

it throws the following error:

Capturing in make/cc mode...
abstractdtypes.c:5:10: fatal error: 'Python.h' file not found
#include <Python.h>
         ^~~~~~~~~~
1 error generated.
Error: the following clang command did not run successfully:
    /opt/infer-linux64-v1.0.0/lib/infer/facebook-clang-plugins/clang/install/bin/clang-9
    @/media/nimashiri/DATA/vsprojects/ICSE23/ml_repos_cloned/numpy/numpy/numpy/core/src/multiarray/infer-out/tmp/clang_command_.tmp.a9d22f.txt
  ++Contents of '/media/nimashiri/DATA/vsprojects/ICSE23/ml_repos_cloned/numpy/numpy/numpy/core/src/multiarray/infer-out/tmp/clang_command_.tmp.a9d22f.txt':
    "-cc1" "-load"...

How can I run infer against individual source files in project having special compilers, e.g. numpy or tensorflow. These compilers do not use make or cmake explicitly.

Thanks.

Solution

  • I figured out how to do that. These are steps to follow:

    1. Compile and build the project, e.g. for numpy when you want to build from source, you have to run python setup.py build_ext --inplace.

    2. When you compile the project, in the terminal, you can see the compilation log. You have to search for compilation commands in the compilation log. For example, when compiling numpy from source, you can see something like this:

    -Inumpy/core/include -Inumpy/core/include/numpy -Ibuild/src.linux-x86_64-3.8/numpy/distutils/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/usr/include/python3.8 -Inumpy/core/src/common -Inumpy/core/src/npymath -c

    1. The previous commands are path to header files needed for compilation. Please note that the previous header files are for numpy/random/src/distributions/logfactorial.c. You have to find the corresponding files either manually or automatically using some parsers.

    2. In this step, you need to run capture command as follows:

    infer capture -- gcc Inumpy/core/include -Inumpy/core/include/numpy -Ibuild/src.linux-x86_64-3.8/numpy/distutils/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/usr/include/python3.8 -Inumpy/core/src/common -Inumpy/core/src/npymath -c numpy/random/src/distributions/logfactorial.c
    1. The previous command will capture compilation commands, now we need to analyze the file for possible errors using analyze command as follows:
    infer analyze -- gcc Inumpy/core/include -Inumpy/core/include/numpy -Ibuild/src.linux-x86_64-3.8/numpy/distutils/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/usr/include/python3.8 -Inumpy/core/src/common -Inumpy/core/src/npymath -c numpy/random/src/distributions/logfactorial.c