Search code examples
pythoncpython-c-apicoroutinecontinuations

Can switching in-and-out PyFrameObjects be a good implementation of continuations?


I'm interested in continuations, specifically in Python's C-API. From what i understand, the nature of continuations requires un-abstracting low-level calling conventions in order to manipulate the call stack as needed. I was fortunate enough to come across a few examples of these scattered here and there. In the few examples i've come across, this un-abstraction is done using either clever C (with assumptions about the environment), or custom assembly.

However, what's cool about Python is that it has its own interpreter stack made up of PyFrameObjects. Assuming single-threaded applications for now, shouldn't it be enough to just switch in-and-out PyFrameObjectss to implement continuations in Python's C-API? Why do these authors even bother with the low-level stuff?


Solution

  • Generators work by manipulating the stack (actually linked list) of frame objects. But that will only help for pure Python code. It won't help you if your code has any C code running. For example, if you are in C code inside an I/O routine, you can't change the Python frame object to get execution to go somewhere else. You have to be able to change the C stack in order to do that. That's what packages like greenlets do for you.