Describe what happened:
For context, i am currently trying to embed python into golang. I came across this module go-python3 and decided to use it. I was originally using the python3.7 variant of the module then decided to upgrade my python version from 3.7 to 3.8.
Read: https://github.com/go-python/cpy3/issues/29#issuecomment-1321199010
I replaced python3.pc in pkgconfig with python-3.8.pc and got the following error when trying to run:
protoc -Isrc/proto --go_opt=module=grpc-golang --go_out=. --go-grpc_opt=module=grpc-golang --go-grpc_out=. src/proto/*.proto
go build -o dest/server ./src/server && PYTHONPATH=.
# grpc-golang/src/server
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0x78): undefined reference to `PyBool_Type'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0x80): undefined reference to `PyByteArray_Type'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0x88): undefined reference to `PyBytes_Type'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0x90): undefined reference to `PyComplex_Type'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0x98): undefined reference to `PyDict_Type'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xa0): undefined reference to `PyExc_ArithmeticError'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xa8): undefined reference to `PyExc_AssertionError'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xb0): undefined reference to `PyExc_AttributeError'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xb8): undefined reference to `PyExc_BaseException'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xc0): undefined reference to `PyExc_BlockingIOError'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xc8): undefined reference to `PyExc_BrokenPipeError'
/usr/bin/ld: /tmp/go-link-2733155445/go.o:(.data+0xd0): undefined reference to `PyExc_BufferError'
:
This are some of the details:
>> python3 --version
3.8.10
>> pkg-config --cflags --libs python3
-I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8
>> echo $PKG_CONFIG_PATH
/usr/lib/x86_64-linux-gnu/pkgconfig/
>> pkg-config --list-all
python-3.8 Python - Build a C extension for Python
zlib zlib - zlib compression library
protobuf-lite Protocol Buffers - Google's Data Interchange Format
bash-completion bash-completion - programmable completion for the bash shell
xkeyboard-config XKeyboardConfig - X Keyboard configuration data
protobuf Protocol Buffers - Google's Data Interchange Format
systemd systemd - systemd System and Service Manager
libdmmp libdmmp - Device mapper multipath management library
python3 Python - Build a C extension for Python
libxcrypt libxcrypt - Extended crypt library for DES, MD5, Blowfish and others
udev udev - udev
shared-mime-info shared-mime-info - Freedesktop common MIME database
python-3.8-embed Python - Embed Python into an application
expat expat - expat XML parser
python3-embed Python - Embed Python into an application
libcrypt libxcrypt - Extended crypt library for DES, MD5, Blowfish and others
Describe what you expected: Should be able to work fine, like its 3.7 variant.
I ran into a similar problem just recently. After searching through a few forums on embedding python, I ran into this one: "Embedding Python in C, linking fails with undefined reference to `Py_Initialize'" 1. I wasn't entirely sure how to do this at first in golang, but I had been doing other online searches on embedded python in golang. Going back through these, I landed at an article call, "Python and Go : Part IV - Using Python in Memory" 2, where it describes in the "Building" section how to specify specific flags to be use in the C building process. Basically, it shows that you can define flags in one of files as such:
/*
#cgo pkg-config: python3
#cgo LDFLAGS: -L/usr/local/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/local/lib -lpython3.8 -lcrypt -ldl -lm -lm
#include "Python.h"
*/
import "C"
Although, according to Python's documentation on compiling a python interpreter embedded in another application, you should get the LDFLAGS via,
python3.7-config --ldflags
for python versions <= 3.7 (I don't know how far back if goes. Also, replace the "python3.7-config" accordingly), or,
python3.10-config --ldflags --embed
for python versions > 3.7 (And once again, replace the "python3.10-config" part with the desired python version)
I then thought to myself, "A version of this header is probably located somewhere in the go-python project and it just needs to be changed", so after looking around, I found this header in the high_level_layer.go file in the project and modified it to match what's shown above.
What's exactly going on here is that golang has native functionality with C, so libraries written in C can work in golang, and the opposite is also true (with enough finagling). There's plenty of articles on this functionality. Here's one from go's official website, 3. And since the base python interpreter is written in C, its header and object files can be referenced at compile time by the golang compiler if the linker flags and other flags are set correctly, so that's exactly what's being specified above.
Also, on a different note, I've tried compiling with a few different go compilers. I've had success with the compiler for versions 1.15.5 and 1.18.1, but haven't with 1.19.5. I don't know what's exactly happening, but I'm not going to try and figure it out right now since all I need currently are features in compiler versions >= 1.18. Lastly, I actually had basically the same problems with python3.7, and these changes fixed these problems too...