I'm using Pyinstaller to pack my python spacy code. I'm using the de_core_news_sm and installed it via pip.
The normal script performs as expected but as soon as it is packaged with pyinstaller it can not find the model [E050] Can't find model 'de_core_news_sm'. It doesn't seem to be a Python package or a valid path to a data directory.
i got for each hook a file:
from PyInstaller.utils.hooks import collect_all
# ----------------------------- blis -----------------------------
data = collect_all('blis')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
from PyInstaller.utils.hooks import collect_all
# ----------------------------- cymen -----------------------------
data = collect_all('cymem')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files("de_core_news_sm")
from PyInstaller.utils.hooks import collect_all
# ----------------------------- preshed -----------------------------
data = collect_all('preshed')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
from PyInstaller.utils.hooks import collect_all
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
from PyInstaller.utils.hooks import collect_all
# ----------------------------- thinc -----------------------------
data = collect_all('thinc')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
and i use the following pyinstaller command:
pyinstaller script.py --hidden-import cmath --hidden-import srsly.msgpack.util
in this post [1]: Can't find SpaCy model when packaging with PyInstaller
the is the solution with the added data files from modules. So how must i change my code in my script.py runtime file and how must i change the hook files?
BTW i'm using: spacy 3.0.6, pyinstaller 4.3 and python 3.8.0 in my virtual environment.
Thank you !
Adding this to my runtime scripts solve the problem. Instead of loading it as a module i'm loading my model from the path
from pathlib import Path
bundle_dir = Path(__file__).parent.absolute()
source_nlp = spacy.load(bundle_dir / "de_core_news_sm")
And the hook file to collect all de_core_news_sm data
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files("de_core_news_sm")