Search code examples
python-c-api

How to create a Python C extension module proposing multiple sub-modules?


I'm attempting to create some complex C extension module for Python.

I have somewhat good results using the official documentation and can import my module with a syntax like

import my_custom_module

But I would also like to have submodules in my C extension like

import my_custom_module

my_custom_module.ClassA # exists in the my_custom_module module

import my_custom_module.subpart1
import my_custom_module.subpart2

my_custom_module.subpart1.ClassB # exists in the my_custom_module.subpart1 module
my_custom_module.subpart2.ClassC # exists in the my_custom_module.subpart2 module

Preferable I would like to keep only one shared library (for ease of build and distribution) and I wouldn't mind if the unique PyMODINIT_FUNC needed to load all of my classes and functions (that's not a problem, I'm not looking at optimizing loading times or to reduce memory consumption here). So this is mostly a syntaxical requirement in order to be able to categorize some classes in some submodules and some classes in other.

Does someone know how to do that, maybe a small example ?

I did a lot of research in the official documentation + by Googling, didn't found an answer.


Solution

  • OK, I found it. You can to use PyImport_AddModule and PyDict_SetItemString in this way (comments are from OpenCV's python binding documentation):

    PyMODINIT_FUNC PyInit_mycustommodule(void) {
      PyObject *main_module = PyModule_Create(&mycustommodule_moduledef);
      if (main_module == NULL) {
        return NULL;
      }
    
      PyObject* main_module_dict = PyModule_GetDict(main_module);
      if (main_module_dict == NULL) {
        return NULL;
      }
    
      PyObject * submodule = PyImport_AddModule("mycustommodule.mysubmodule");
      if (submodule == NULL) {
        return NULL;
      }
    
      /// Populates parent module dictionary. Submodule lifetime should be managed
      /// by the global modules dictionary and parent module dictionary, so Py_DECREF after
      /// successfull call to the `PyDict_SetItemString` is redundant.
      if (PyDict_SetItemString(main_module_dict, "mysubmodule", submodule) < 0) {
          return NULL;
      }
    
      return main_module;
    }