I am getting the following error while loading spacy NER model from the pickle file.
self.model = pickle.load(open(model_path, 'rb'))
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py", line 1496, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Program Files\JetBrains\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc) File
"C:\Projects\pythonworkspace\invoice_processing_prototype\invoice_data_extractor_notebook.py", line 101, in
<module>
extractor = InvoiceDataExtractor(model_dir_path, input_file_paths[0], config_path) File
"C:\Projects\pythonworkspace\invoice_processing_prototype\invoicedataextractor.py", line 27, in
__init__ self.spatial_extractor = SpatialExtractor(model_dir_path, config_path)
File "C:\Projects\pythonworkspace\invoice_processing_prototype\spatialextractor.py", line 54, in __init__
self.inv_date = Model(f{self.model_dir_path}\\invoice_date_with_corrected_training_data_and_line_seperator_21_07_2022.pkl", File
"C:\Projects\pythonworkspace\invoice_processing_prototype\spatialextractor.py", line 34, in __init__
self.model = pickle.load(open(model_path, 'rb'))
File "stringsource", line 6, in spacy.pipeline.trainable_pipe.__pyx_unpickle_TrainablePipe _pickle.PickleError:
Incompatible checksums (0x417ddeb vs (0x61fbab5, 0x27e6ee8, 0xbe56bc9) = (cfg, model, name, scorer, vocab))
I am getting this error while running the following line of code:
self.model = pickle.load(open(model_path, 'rb'))
I have trained the NER model using the spacy version 3.1.2 and I recently upgraded the spacy to the latest 3.4. The error might be because of some version incompatibilities. If that is the case can someone confirm if is it possible to load spacy NER model trained on spacy version '3.1.2' can be loaded on the upgraded spacy '3.4'
Environment
Operating System: Windows 10
Python Version Used: 3.10
spaCy Version Used while training: 3.1.2
spaCy Version Used while prediction: 3.4
I found one workaround to this error.
While on spacy 3.1.2
I have loaded the pickle model and saved it using
model = pickle.load(open(filepath, 'rb'))
model.to_disc('my_model')
Then I have updated the spacy version to latest spacy 3.4.4
and reloaded the 'my_model' this time.
model = spacy.load("my_model")
pickle.dump(model, open(filepath, 'wb'))
In that case it worked for me without the error.