Search code examples
iosmemory-leaksavcapturesessioncore-image

CIImage and CIDetector use with AVCaptureOutput memory leak


I'm using a CIContext, CIDetector, and CIImage to detect rectangles in a vImage_Buffer derived from samples in captureOutput:didOutputSampleBuffer:fromConnection:. It seems that either the detector or the CIImage is retaining memory and it cannot be released. Here is the code in question - skipping over this code shows memory held constant, otherwise in increases until crashing the app:

// ...rotatedBuf and format managed outside scope

// Use a CIDetector to find any potential subslices to process
CVPixelBufferRef cvBuffer;
vImageCVImageFormatRef cvFormat = vImageCVImageFormat_CreateWithCVPixelBuffer(pixelBuffer);
CVPixelBufferCreate(kCFAllocatorSystemDefault, rotatedBuf.width, rotatedBuf.height, kCVPixelFormatType_32BGRA, NULL, &cvBuffer);
CVPixelBufferLockBaseAddress(cvBuffer, kCVPixelBufferLock_ReadOnly);
err = vImageBuffer_CopyToCVPixelBuffer(&rotatedBuf, &format, cvBuffer, cvFormat, NULL, kvImageNoFlags);
CVPixelBufferUnlockBaseAddress(cvBuffer, kCVPixelBufferLock_ReadOnly);
if (![self vImageDidError:err]) {
    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:cvBuffer];
    NSArray *feats = [self.ciDetector featuresInImage:ciImage options:nil];
    if (feats && [feats count]) {
        for (CIFeature *feat in feats) {
            // The frame is currently in image space, so we must convert it to a unitless space like the other rects.
            CGRect frame = feat.bounds;
            CGRect clip  = CGRectMake(frame.origin.x / rotatedBuf.width, frame.origin.y / rotatedBuf.height,
                                      frame.size.width / rotatedBuf.width, frame.size.height / rotatedBuf.height);
            rects = [rects arrayByAddingObject:[NSValue valueWithCGRect:clip]];
        }
    }
}
CVPixelBufferRelease(cvBuffer);
vImageCVImageFormat_Release(cvFormat);

Other answers seem to suggest wrapping in an autorelease pool or create a new CIDector each frame, but neither affect the memory use.

CIDetector isn't releasing memory

CIDetector won't release memory - swift

Edit: switching the dispatch queue to one other than dispatch_main_queue seemed to have cleared the memory issue and keeps the UI responsive.


Solution

  • I figured out a different solution - in my case I was running all my processing on the main dispatch queue. What fixed the situation was creating a new queue to run the processing in. I realized this may be the case when the majority of my CPU time was spent on the call to featuresInImage:options:. It doesn't explain what caused the memory issue, but now that I'm running in a separate queue, memory is nice and constant.