I'm looking for a Python framework that will enable me to play video as well as draw on that video (for labeling purposes).
I've tried Pyglet, but this doesn't seem to work particularly well - when drawing on an existing video, there is flicker (even with double buffering and all of that good stuff), and there doesn't seem to be a way to get the frame index in the video during the per-frame callback (only elapsed time since the last frame).
Try a Python wrapper for OpenCV such as ctypes-opencv. The C API reference is here, and the wrapper is very close (see docstrings for any changes).
I have used it to draw on video without any flicker, so you should have no problems with that.
A rough outline of calls you need: