Search code examples
pythonpipapple-m1pypi

Installing ssdeep package from PyPi on M1 Macbook


The Goal

Install ssdeep PyPi package on a M1 Macbook Pro.

The Problem

When I run pip install ssdeep I get 2 errors

The first error is caused because fuzzy.h cannot be found.

warnings.warn(
  running egg_info
  creating /private/var/folders/0f/4c82tsj50n10zqq89fndcslc0000gn/T/pip-pip-egg-info-ai0atrdv/ssdeep.egg-info
  writing /private/var/folders/0f/4c82tsj50n10zqq89fndcslc0000gn/T/pip-pip-egg-info-ai0atrdv/ssdeep.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/0f/4c82tsj50n10zqq89fndcslc0000gn/T/pip-pip-egg-info-ai0atrdv/ssdeep.egg-info/dependency_links.txt
  writing requirements to /private/var/folders/0f/4c82tsj50n10zqq89fndcslc0000gn/T/pip-pip-egg-info-ai0atrdv/ssdeep.egg-info/requires.txt
  writing top-level names to /private/var/folders/0f/4c82tsj50n10zqq89fndcslc0000gn/T/pip-pip-egg-info-ai0atrdv/ssdeep.egg-info/top_level.txt
  writing manifest file '/private/var/folders/0f/4c82tsj50n10zqq89fndcslc0000gn/T/pip-pip-egg-info-ai0atrdv/ssdeep.egg-info/SOURCES.txt'
  src/ssdeep/__pycache__/_ssdeep_cffi_a28e5628x27adcb8d.c:266:14: fatal error: 'fuzzy.h' file not found
      #include "fuzzy.h"
                ^~~~~~~~~
  1 error generated.
  Traceback (most recent call last):
    File "/Users/user/proj/.venv/lib/python3.11/site-packages/setuptools/_distutils/unixccompiler.py", line 186, in _compile
      self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
    File "/Users/user/proj/.venv/lib/python3.11/site-packages/setuptools/_distutils/ccompiler.py", line 1007, in spawn
      spawn(cmd, dry_run=self.dry_run, **kwargs)
    File "/Users/user/proj/.venv/lib/python3.11/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
      raise DistutilsExecError(
  distutils.errors.DistutilsExecError: command '/usr/bin/clang' failed with exit code 1

The second error has to do with setuptools.installer being deprecated. I'm not sure this is all that important though. I think resolving the first error would resolve this one as well.

/Users/user/proj/.venv/lib/python3.11/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.

Attempted Solutions

Solution 1: Install SSDeep with Homebrew brew install ssdeep

Result: pip install ssdeep has the same error about fuzzy.h missing

Solution 2: Use the prepackaged version of SSDeep BUILD_LIB=1 pip install ssdeep

Result: The error about fuzzy.h goes away but the second error regarding setuptools.installer being deprecated remains.

References


Solution

  • I found a solution. Essentially what's going on is that Homebrew installs ssdeep in a location that the ssdeep PyPi package is not expecting. You can point the PyPi package to the correct locations with the following steps.

    1: Install ssdeep with homebrew brew install ssdeep

    2: List homebrew directories for ssdeep brew ls ssdeep This produces output like

    /opt/homebrew/Cellar/ssdeep/2.14.1/bin/ssdeep
    /opt/homebrew/Cellar/ssdeep/2.14.1/include/ (2 files)
    /opt/homebrew/Cellar/ssdeep/2.14.1/lib/libfuzzy.2.dylib
    /opt/homebrew/Cellar/ssdeep/2.14.1/lib/ (2 other files)
    /opt/homebrew/Cellar/ssdeep/2.14.1/share/man/man1/ssdeep.1
    

    3: Set the LDFLAGS environment variable to the path of the ssdeep lib directory from the output in step 2.

    export LDFLAGS="-L/opt/homebrew/Cellar/ssdeep/2.14.1/lib"
    

    4: Set the C_INCLUDE_PATH environment variable to the path of the ssdeep include directory from the output in step 2.

    export C_INCLUDE_PATH=/opt/homebrew/Cellar/ssdeep/2.14.1/include
    

    5: Install ssdeep from PyPi pip install ssdeep