I am embedding Python in C++.
I have a working C++ Python extension object.
The only thing wrong is that if I set tp_dealloc to a custom function it never gets called.
I would have thought Py_Finalize() would trigger this, or maybe terminating the program. But no.
Could anyone suggest why tp_dealloc isn't getting hit?
I believe the problem here was one of reference counting.
PyType_Ready() fills various tp_* fields depending on the bases of your type.
One of these is tp_alloc, which I have set to 0. Its doc says the refcount is set to 1 and the memory block is zeroed.
Every instance Python creates of this type, a new PyObject get added to the appropriate Python Dictionary. If it is a module level variable, this is the module's dictionary.
When the dictionary is destroyed, it DECREF-s contained objects. Now the refcount will be 0, and tp_dealloc will get run.
It appears that in my code I was performing an extra INCREF somewhere and the object was never getting garbage collected.
It seems that (unless you compile with the specific flag) Python has no linked list that would allow it to track all of its objects. So we can't assume that Py_Finalize() will clear up. It won't!
Instead, every object is held in the dictionary for its containing scope, and so on back to the module dictionary. When this module dictionary is destroyed, the destruction will creep outwards through all the branches.