Search code examples
pythonincludepyd

Is there a way to import a .pyd extension file as a simple include in a python file?

Hi there wise people of stack. I'm having trouble with importing pyd files as python objects.

The story:

I have an internal repo on gitlab that runs python files as well as C++ files. the repo uses pybind for both languages to speak to one another. The whole project is built with CI/CD and the artefact I have access to are .pyd extension files.

What I was given as a task would be to access some .pyd files (in different folders) with a single python script and access their classes (encoded inside this .pyd file) to mock them using python.

The problem:

What I was told was that I would need a simple include to be able to access the .pyd as an object through python just like you would with a library. However, I came across errors in the whole process. I have gone through this post and this one, but it seems that none of them works for me.

What was tried:

The first thing I did was start a remote folder with a single .pyd file from the project(let's call it SomeClass.pyd). I then created a python file test.py in the same directory as the pyd file. The whole architecture looks like the following:

|--folder
      |--SomeClass.pyd
      |--test.py

Then, in the test.py file, I tried running

import SomeClass.pyd

import SomeClass

import SomeClass.pyd as sc

from SomeClass.pyd import *

from SomeClass import *

which all yielded the same following error: ImportError: dynamic module does not define module export function

Now, I know that pyd files are similar to dlls, but I was told multiple time that a simple import would let me access the object information without needing anything in particular.

I recall reading about adding the PYTHONPATH before launching the whole process. However, I need that file to access the pyd without having to add any variable to the path as I will likely not always have access rights to the PYTHONPATH.

The project is quite big, so I'm trying to keep it bare minimum, but if you need more info, I'll try to give some more.

Thank you for your feedback!

Solution

  • Alright, after some time and a lot of researching, I found the weird answer for the problem that occured. I really hope it will help anyone encountering the same issue.

    The problem was caused by the fact that pycharm has sometimes issues with the whole dynamic import.

    First problem: dynamic import

    This was solved simply by going on pycharm --> files --> invalidate cache and then tick "Clear file system cache and Local History" as well as "Clear VCS Log caches and indexes". You should then be prompted to reboot.

    I also add a note that even after fixing the issue, sometime, for no apparent reason, I still have to invalidate cache again.

    Second problem: venv

    Once rebooted, you might be able to import manually the path to your pyd file, but you probably won't be able to auto complete. What solved this for me was compiling manually the code responsible for the pyd in order to generate a wheel. In my case, I used poetry:

    poetry build

    Once the wheel was created, I did a manual pip install of the wheel created by the poetry build to install it directly into the venv:

    pip install dist/the_name_of_your_wheel_file.whl

    These steps were the ones to fix my problem. I hope this will help anyone encountering the same problem!