Capturing video and displaying images using captureOutput:captureOutput didOutputSampleBuffer:sampleBuffer fromConnection:connection

Question

I am trying to better understand the AVFoundation framework along with the various Core xxxx frameworks so I decided to try a simple video capture and see if I can output as images to the UI. I looked at the rosyWriter code as well as documentation but with no answer. So:

I have the standard capture session code to add input and output. The following is relevant to the question:

//moving the buffer processing off the main queue
dispatch_queue_t bufferProcessingQueue=dispatch_queue_create("theBufferQueue", NULL);
[self.theOutput setSampleBufferDelegate:self queue:bufferProcessingQueue];
dispatch_release(bufferProcessingQueue);

And then the delegate:

-(void) captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
       fromConnection:(AVCaptureConnection *)connection
    {

    CVPixelBufferRef pb = CMSampleBufferGetImageBuffer(sampleBuffer);

    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pb];
    CGImageRef ref = [self.theContext createCGImage:ciImage fromRect:ciImage.extent];

    dispatch_async(dispatch_get_main_queue(), ^{
        self.testBufferImage.image= [UIImage imageWithCGImage:ref scale:1.0 orientation:UIImageOrientationRight];
    });
}

Questions:

1- I am guessing that as I did above, we should always set the delegate to run on a separate queue as I did above and not the main queue. Correct?

2- In conjunction, in the delegate method, any calls that deal with the UI have to be put back to the main queue like I did. Correct?

3- When I run this code, after about 5-10 seconds, I get a "Received memory warning" error and the app shuts. What could cause this?

Answer 1

1) Generally yes you should. You could run it on the main queue, but this can cause issues with UI responsiveness among other things.

2) Correct.

3) You are creating a series of CGImageRefs. Where are you releasing them?

For performance reasons you should probably use OpenGL if you need fine control over the rendering of the video. Otherwise you can use AVCaptureVideoPreviewLayer for an easy way to get a preview.