Cannot import Identical Github Fork of Python CoreMLTools

I am having some issues using CoreMLTools, so I am going to try to debug it using my own Github repo. I forked the original project and modified nothing, but I get an error when I try to import it.

This works perfectly fine:

!pip install coremltools
import coremltools as ct

But this:

!pip install git+https://github.com/[Owner]/[Repo Name].git 
import coremltools as ct # Identical Forked Copy of CoreMLTools

has the following error:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-4-1b9428219e3f> in <module>
      1 #!pip install coremltools
      2 get_ipython().system('pip install git+https://github.com/[Owner]/[Repo].git')
----> 3 import coremltools as ct

8 frames
/usr/local/lib/python3.7/dist-packages/coremltools/converters/mil/frontend/milproto/load.py in <module>
     25 from coremltools.converters.mil.mil.block import curr_block, curr_opset_version
     26 from coremltools.converters.mil.mil.ops.registry import SSAOpRegistry as _SSAOpRegistry
---> 27 from coremltools.libmilstoragepython import _BlobStorageReader as BlobReader
     28 from coremltools.proto import (
     29     MIL_pb2 as pm,

ModuleNotFoundError: No module named 'coremltools.libmilstoragepython'

EDIT: The following link seems to be a C++ file that does something with the module that can't be found. https://github.com/apple/coremltools/blob/973eae67f2f273a29e80a9b009987516a070a58b/milstoragepython/MilStoragePython.cpp

Solution

coremltools does not build the libmilstoragepython source if you try to do a pip install of its source code - it would run setup.py which doesn't do any of the C++ compilation steps.

Instead, what you have to do is manually build the entire project and install the wheel file it generates. The main documentation for this is from their BUILDING.md.

It looks like you're on some flavor of Linux, so you're probably going to need the following dependencies:

C++ compiler with C++17 support (gcc-8 or newer)
conda (either miniconda or Anaconda)
cmake (either through package manager or installed from pip/conda)
zsh
libuuid-devel (your distro might call this uuid-devel or similar)

# You need conda installed and configured for zsh
# https://docs.conda.io/en/latest/miniconda.html

# substitute your own coremltools fork here
git clone https://github.com/apple/coremltools/

cd coremltools

# You can specify whatever version of Conda-supported Python you want this for in the range of...
# 3.7, 3.8, 3.9
# I also assume you want debug mode
zsh -i scripts/build.sh --python=3.7 --dist --debug

# this will output many cmake messages, but hopefully it succeeds

After this, you will have a directory called build/dist that contains a .whl file that can be installed

ls build/dist
coremltools-6.0b2-cp37-none-manylinux1_x86_64.whl