I am trying to find out how Cython compiles .pyx into .cpp files, and how to find the exact piece of C++ translation of my .pyx file.
My file is a simple helloworld.pyx file
print("Hello world")
I use my setup.py as follows from the official Cython guide, which is as follows:
from Cython.Build import cythonize
extensions = [
Extension(
"helloworld",
sources=["helloworld.pyx"],
language="c++"
),
]
setup(ext_modules = cythonize(extensions))
After running python3 setup.py build_ext --inplace
This very nicely produces both a .cpp and .so file. However, I get approximately 3000 lines of code in my .cpp file, and most of it are definitions and the like. I was rather happy when I saw that the file contained the following:
/* Implementation of 'helloworld' */
static const char __pyx_k_end[] = "end";
static const char __pyx_k_file[] = "file";
static const char __pyx_k_main[] = "__main__";
static const char __pyx_k_name[] = "__name__";
static const char __pyx_k_test[] = "__test__";
static const char __pyx_k_print[] = "print";
static const char __pyx_k_Hello_world[] = "Hello world";
static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback";
static PyObject *__pyx_kp_s_Hello_world;
static PyObject *__pyx_n_s_cline_in_traceback;
static PyObject *__pyx_n_s_end;
static PyObject *__pyx_n_s_file;
static PyObject *__pyx_n_s_main;
static PyObject *__pyx_n_s_name;
static PyObject *__pyx_n_s_print;
static PyObject *__pyx_n_s_test;
/* Late includes */
However, this is not valid .cpp. Is it possible to generate valid C++ code in the process of compilation and output this piece of code somehow? I am not able to copy the entire file, as it is too long, sadly.
EDIT:
A very concrete example of what I wish to achieve is the exact things these guys are doing: Generating SIMD instructions from Cython code
They write Cython, use the command as I, and then get some clean C code which is compilable.
First thing to make very clear: Cython does not create short self-contained code that's independently compilable. It's designed to be compiled as part of a Cython module with all the 3000+ lines of utility code that surrounds it. This is especially true for code that interacts with Python a lot (for example your print statement).
So if you are expecting to be able to copy a short function of C++ code and run it independently the answer is "you can't".
Second:
However, this is not valid .cpp. [...]
That's just wrong. It's completely valid C++. It isn't standalone (it depends on Python.h) but it is a set of declarations of global constants.
Third (actually directly answering your question) - by default Cython embeds your input Cython code into comments into your file. That makes it very easy to find the C/C++ code that matches up with the code you have it just by searching for a particular line (remember that one line may generate code in more than one place though, particularly for function definitions).
The key line in your file is
/* "hw.py":1
* print("Hello world") # <<<<<<<<<<<<<<
*/
if (__Pyx_PrintOne(0, __pyx_kp_s_Hello_world) < 0) __PYX_ERR(0, 1, __pyx_L1_error)
Note that it depends on the Cython utility-code function __Pyx_PrintOne and the global string constant __pyx_kp_s_Hello_world. I.e. not standalone.
Another useful tool is the cython -a command-line option that creates an annotated HTML file showing how your input code links up with C/C++ code.