recording video in real-time with OpevCV VideoWriter

Question

I have a few OpenCV projects that analyze video over USB, and in certain conditions must record the video stream to a file. People using my software complain that 10+ minute recordings yield video files that are about 20 seconds longer than they should be.

I'm using openCV's VideoWriter. Iv'e tried things like setting CV2_CAP_PROP_FPS to a very low setting, and iv'e tried getting the average frame rate over a few seconds to find a good setting for my frame rate of the output file. Still not close enough to real time for my needs.

Does anyone know of a good way to make sure my video is recording close to real time? Should I use something like time.sleep (in python) to cap my framerate? Or is there a better way to do this?

Answer 1

I wrote a real-time tracker (for neurobio research), which I use with many webcams, and noticed that some of my cameras do not adhere to frame rates very precisely; they can be somewhat fast or slow. To save "reasonably correct" videos, the code capturing frames from the webcam calls my VideoWriter's write(frame) method, which puts the frame into a queue, and the frames get retrieved by a separate "writer" thread that writes frames at a constant rate. If the writer thread finds the queue empty when the next frame is to be written, it repeats the last frame. If the writer thread finds more than one frame in the queue, it writes the most recent frame and discards the other ones. (One could implement this without a separate thread, but for a real-time tracker, it is nice to have write(frame) return quickly.)

Below excerpts from my code. They will not run verbatim, but show what I just described. (I plan to put the real-time tracker on GitHub in the next couple of weeks.)

class VideoWriter:

  def __init__(self, fn=None, fcc='MJPG', fps=7.5):
    ...
    self.fcc, self.fps, self.dt = fcc, fps, 1./fps
    self.q, self._stop, self.n = Queue.Queue(), False, 0
    self.wrtr = threading.Thread(target=self._writer)
    self.wrtr.start()

  def _writer(self):
    frm = firstTime = vw = None
    while True:
      if self._stop:
        break
      # get most recent frame
      while not self.q.empty():
        frm = self.q.get_nowait()
      if frm is not None:
        if vw is None:
          vw = cv2.VideoWriter(self.fn+self._EXT, cvFourcc(self.fcc),
            self.fps, imgSize(frm), isColor=numChannels(frm)>1)
          firstTime = time.time()
        vw.write(frm)
        self.n += 1
      dt = self.dt if firstTime is None else max(0,
        firstTime + self.n*self.dt - time.time())
      time.sleep(dt)

  # write frame; can be called at rate different from fps
  def write(self, frm):
    self.q.put(frm)