Search code examples
pythonc++python-embedding

PyEval_CallObject and PyObject_CallObject return a null object


This is a followup question concerning this one. As pointed in the former question comments, my initial attempt for importing the module was completely wrong, and instead I should have done this :

auto module = PyImport_ImportModule("Package1.Module1");

so I did so :

auto module = PyImport_ImportModule("FV.F_V");

After doing this, now I get much more meaningful output as you can see:

module: 0000021306FF9B38
size: 30
FaceVerification: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Image: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Path: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__builtins__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__cached__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__doc__: FV.F_V
__file__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\F_V.py
__loader__: FV
__name__: FV.F_V
__package__: FV
__spec__: FV
__warningregistry__ : C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
align_face: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
cv2: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
data_transforms: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
get_central_face_attributes: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
math: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
nn: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
np: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
os: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
pickle: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
plt: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet101: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet18: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet50: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
scipy: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
time: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
torch: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
tqdm: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
transforms : C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc

However, when ever I try to use a class defined in the module (i.e FaceVerificationand instantiate it, it fails. The class seems OK, but the calls that try to instantiate the object, return a null object!: I tried both PyEval_CallObject and PyObject_CallObject:

auto python_class = PyDict_GetItemString(pdict, "FaceVerification");

if (PyCallable_Check(python_class)) 
{
    object = PyEval_CallObject(python_class, args);
    # or 
    //object = PyObject_CallObject(python_class, args);
    //Py_DECREF(python_class);
}
else 
{
    cout << "Cannot instantiate the Python class" << endl;
    //Py_DECREF(python_class);
    //return 1;
}

to no avail. This is strange, I therefore added a new CONSTANT field to the F_V.py and tried to see if I can access that (to see if the module was indeed valid). It actually successfully retrieved the constant. However, I noticed, on the re-running of the c++ application, now when I print the map which holds the output of PyModule_GetDict(module), I see a different result :

module: 000001F7E07E8C28
size: 31
CONSTANT: THIS IS A TEST. REMOVE IT
FaceVerification: THIS IS A TEST. REMOVE IT
Image: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
Path: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__builtins__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__cached__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
__doc__: FV.F_V
__file__: C:\Users\Master\Anaconda3\Lib\site-packages\FV\F_V.py
__loader__: FV
__name__: FV.F_V
__package__: FV
__spec__: FV
__warningregistry__: THIS IS A TEST. REMOVE IT
align_face: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
cv2: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
data_transforms: THIS IS A TEST. REMOVE IT
get_central_face_attributes: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
math: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
nn: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
np: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
os: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
pickle: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
plt: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet101: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet18: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
resnet50: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
scipy: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
time: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
torch: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
tqdm: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc
transforms: C:\Users\Master\Anaconda3\Lib\site-packages\FV\__pycache__\F_V.cpython-36.pyc

most notably the FaceVerification which should be a class, is not showing the CONSTANT field's value!

What am I doing wrong here?


Solution

  • As it turns out(many thanks to @Scheff), the reason why the object was null, was because an exception was happening inside the constructor and thus resulted in the object being null. The reason for the exception was a malformed path and problematic (wrong) arguments.
    basically I had set the arguments like this :

    auto model_name = "r189";
    auto model_checkpoint_path = PyUnicode_DecodeFSDefault("L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar");
    auto align_fn = Py_None;
    auto img_bank_folder_root = PyUnicode_DecodeFSDefault("L:\\test_procs\\data\\facebank_2");
    auto cache_folder = Py_None;
    auto postfix = Py_None;
    auto rebuild_cache = Py_False;
    auto use_jit = Py_False;
    auto threshold = 65; 
    auto device = "cpu";
    
    

    and the arguments thus like this :

    auto args = Py_BuildValue("sOOOOONNiN", model_name, model_checkpoint_path, align_fn,
                                      img_bank_folder_root, cache_folder, postfix, 
                                      rebuild_cache, use_jit, threshold, device);
    

    which was wrong. I changed them to this :

    auto model_name = "r189";
    auto model_checkpoint_path = "L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar";
    auto align_fn = "";
    auto img_bank_folder_root = "L:\\test_procs\\data\\facebank_2";
    auto cache_folder = "";
    auto postfix = "";
    auto rebuild_cache = 0;
    auto use_jit = 0;
    auto threshold = 65; 
    auto device = "cpu";
    
    

    and simplified the args like this :

    auto args = Py_BuildValue("ssssssiiis", model_name, model_checkpoint_path, align_fn,
                              img_bank_folder_root, cache_folder, postfix, 
                              rebuild_cache, use_jit, threshold, device);
    

    after doing all of this, I still faced the exception and found out : model_checkpoint_path needed a fixed as its path lacked a single \ in its path:

    auto model_checkpoint_path = "L:\\test_procs\\trained_models\BEST_checkpoint_r189.tar";  
    

    fixing it :

    auto model_checkpoint_path = "L:\\test_procs\\trained_models\\BEST_checkpoint_r189.tar";
    

    thus solved everything.