Search code examples
pythonboost-pythonpython-modulesynthetic

synthesized submodule: from A import B (ok) vs. import A.B (error)?


I have a module modA, which contains a synthesized submodule modB (created with PyModule_New); now importing the module:

  1. from modA import modB it is OK
  2. import modA.modB fails.

What am I missing?

  • modA.cpp (using boost::python, but it would be very likely the same with pure c-API of python):

    #include<boost/python.hpp>
    namespace py=boost::python;
    
    BOOST_PYTHON_MODULE(modA){
       py::object modB=py::object(py::handle<>(PyModule_New("modB")));
       modB.attr("__file__")="<synthetic>";
       py::scope().attr("modB")=modB;
    };
    
  • compile with (g++ instead of clang++ works the same)

    clang++ -o modA.so modA.cpp -fPIC -shared  -lboost_python `pkg-config python --cflags --libs`
    
  • test.py:

    import sys
    sys.path.append('.')
    from modA import modB
    import modA.modB
    
  • python test.py (note the first import is just fine, it is the second one which fails):

    Traceback (most recent call last):
      File "test.py", line 4, in <module>
        import modA.modB
    ImportError: No module named modB
    

Solution

  • Thanks to this answer, I found the solution, which consists in sys.modules['modA.modB']=modB, but written in c++ in the module initialization function:

    #include<boost/python.hpp>
    namespace py=boost::python;
    
    BOOST_PYTHON_MODULE(modA){
       py::object modB=py::object(py::handle<>(PyModule_New("modA.modB")));
       modB.attr("__file__")="<synthetic>";
       py::scope().attr("modB")=modB;
    
       // this was the missing piece: sys.modules['modA.modB']=modB
       py::extract<py::dict>(py::getattr(py::import("sys"),"modules"))()["modA.modB"]=modB;
    };