Let's say I have a simple C++ class:
class Widget
{
public:
Widget() :
x(1)
{ }
void sleep()
{
sleep(x);
}
private:
int x;
};
Widget::sleep
blocks, so I would like to free up the GIL so that Python can do some other stuff, so I wrapped Widget::sleep
in a function like this:
static void pyWidgetSleep(Widget& self)
{
PyThreadState* state = PyEval_SaveThread();
self.sleep();
PyEval_RestoreThread(state);
}
For completeness, the binding code looks like this:
BOOST_PYTHON_MODULE(libnative)
{
boost::python::class_<Widget>("Widget")
.def("sleep", &pyWidgetSleep);
}
I have a simple Python script that looks like this:
import libnative
import threading
class C:
def __init__(self):
self.x = libnative.Widget()
self.sleeping = False
def foo(self):
self.sleeping = True
self.x.sleep()
c = C()
t = threading.Thread(target=c.foo)
t.start()
while not c.sleeping:
pass
del c.x
Do all participants in an expression get their ref count incremented for the duration of the expression?
The only referant to c.x
when used in C.foo
is c
, which the line after t.start()
conveniently deletes.
Since pyWidgetSleep
releases the GIL, del c.x
might decrement the last reference to the Widget
instance, causing undefined behavior.
I can't make this break on my machine, which seems to be a good indication that it works as I would expect, but reference-counting isn't documented to this degree of clarity
(or, at the very least, I can't seem to find it). The relevant portion of CPython appears to be PyEval_EvalCodeEx
, which looks like it applies Py_INCREF
to all arguments
involved in a function call, but I could be totally off on that one.
The answer to this question is yes, it is safe. This happens as a consequence of how member functions are called in Python. Consider if Widget
was implemented in Python:
class Widget:
def __init__(self):
self.x = 1
def sleep(self):
os.sleep(self.x)
Notice how the first parameter to Widget.sleep
is self
? The reference count is incremented for the duration of the call! It seems obvious in retrospect...