How do I import the pybind module I incorporated into my python module?

Question

I have created a python module that has the following structure (loosely based on this example):

module_name
├── LICENSE
├── README.md
├── module_name
│   ├── __init__.py
│   └── submodule_name
│       ├── __init__.py
│       └── submodule_name.py
├── setup.py
└── src
    └── sq.cpp

I can run setup.py and install this module without a problem. I can also import the pure python modules:

import module_name
import module_name.submodule_name

For sq.cpp I have the following in setup.py

from pybind11 import get_cmake_dir
from pybind11.setup_helpers import Pybind11Extension, build_ext


ext_modules = [
    Pybind11Extension("sq",
    ["src/sq.cpp"],),
]

sq.cpp defines the pybind module as

PYBIND11_MODULE(sq, m)
{
    m.doc() = "a function";
    m.def("sq", &sq, "a function");
}

At this point I'm confused about how to load the module in sq.cpp. I have tried import sq, import module_name.sq, import sq.sq but all result in a ModuleNotFoundError. What am I missing? How am I supposed to import the pybind module? Do I have the structure wrong?

Answer 1

Your build should fail as the names of the module and function are the same. (Look at sq.cpp) When you define a module with PYBIND11_MODULE(sq, m), the sq becomes a local variable and &sq will reffer to another object, but not to the sq function. So you can rename the name of the function to keep the module name as sq.

#include <pybind11/pybind11.h>

void _sq() {}

PYBIND11_MODULE(sq, m) {

    m.doc() = "a function";
    m.def("sq", &_sq, "a function");
}

Compare the following setup.py with yours

import sys

from pybind11 import get_cmake_dir
from pybind11.setup_helpers import Pybind11Extension, build_ext
from setuptools import setup

__version__ = "0.0.1"

ext_modules = [
    Pybind11Extension("sq",
                      ["src/sq.cpp"],
                      define_macros=[('VERSION_INFO', __version__)],
                      ),
]

setup(
    name="sq",
    version=__version__,
    ext_modules=ext_modules,
    cmdclass={"build_ext": build_ext},
    zip_safe=False,
    python_requires=">=3.6",
)

Install the module

Redirect to the parent directory of the sq module directory and execute the following command.

python3 -m pip install ./sq

Import the module

import sq

sq.sq()

Edit notes

I'm a bit confused because I already have a setup section with name="module_name". How do I embed the sq module inside the whole module_name module?

You should change the module name by changing

• Pybind11Extension("sq", ...) to Pybind11Extension("module_name", ...)
• setup(name="sq", ...) to setup(name="module_name", ...)
• PYBIND11_MODULE(sq, m) to PYBIND11_MODULE(module_name, m)

After reinstalling the module with the command above, you can use the sq function in these ways.

import module_name
from module_name import sq

module_name.sq()
sq()