numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Question

I want to call my Python module from the Matlab. I received the error:

Error using numpy_ops>init thinc.backends.numpy_ops

Python Error:

 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject.

The Python script is as follows

import spacy
def text_recognizer(model_path, text):
try:
    # Load the trained model
    nlp = spacy.load(model_path)
    print("Model loaded successfully.")
    
    # Process the given text
    doc = nlp(text)
    ent_labels = [(ent.text, ent.label_) for ent in doc.ents]
        return ent_labels

The Matlab script is as follows

% Set up the Python environment
        pe = pyenv;
        py.importlib.import_module('final_output');
        
        % Add the directory containing the Python script to the Python path
        path_add = fileparts(which('final_output.py'));
        if count(py.sys.path, path_add) == 0
            insert(py.sys.path, int64(0), path_add);
        end
        % Define model path and text to process
        model_path = 'D:\trained_model\\output\\model-best';
        text = 'Roses are red';
        % Call the Python function
        pyOut = py.final_output.text_recognizer(model_path, text);
        % Convert the output to a MATLAB cell array
        entity_labels = cell(pyOut);
        disp(entity_labels);

I found one solution to update Numpy, what I did, but nothing changed. I am using Python 3.9 and Numpy version 2.0.0

The error was received when I tried to call the Python module using a Matlab script.

How can I fix the issue?

Answer 1

The reason is that pandas defines its numpy dependency freely as "anything newer than certain version of numpy". The problem occured, when numpy==2.0.0 has been released on June 16th 2024, because it is no longer compatible with your pandas version.

The solution is to pin down the numpy version to any before the 2.0.0. Today it could be (this is the most recent numpy 1 release):

numpy==1.26.4

To be added in your requirements or to the pip command you use (but together with installing pandas).

Nowadays pip is very flexible and can handle the issue flawesly. You just need to ask it to install both pandas and numpy of given versions in the same pip install invocation.