GPUImage - use NSMutableArray to apply filters in series

Question

I'm trying to build a dynamic system for applying multiple GPUImage filters in series.

If I statically define all of the filters like so:

[sourceImage addTarget:filter1];
[filter1 addTarget:filter2];
[filter2 addTarget:filter3];

... and so on, this works fine.

Since I want to have a variable number of filters though, this gets cumbersome. I figured I'd try and use a mutableArray and store the filters to perform this. Unfortunately I'm getting the error:

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Tried to overrelease a framebuffer, did you forget to call -useNextFrameForImageCapture before using -imageFromCurrentFramebuffer?'

Researching this error has suggested that the problem is that the filters are not 'strong' references anymore, and that once the array is iterated upon, they're destroyed.

If I'm right in this assumption, is there a way to rewrite this so that the references aren't destroyed?

Here's my category implementation:

@implementation UIImage (FilterEngine)

- (UIImage*)imageProcessedWithFilterEngine:(FilterProperties*)properties
{

    // Mutable array of filters, for dynamic iteration through only those filters which require applying
    NSMutableArray* filtersList = [NSMutableArray new];

    GPUImagePicture *source = [[GPUImagePicture alloc] initWithImage:self];

    GPUImageCustomFilterEngine* filterEngine;
    if (properties.lookupImageName)
    {
        filterEngine = [[GPUImageCustomFilterEngine alloc] initWithLookupImageName:properties.lookupImageName];
        [filtersList addObject:filterEngine];
    }

    GPUImageBrightnessFilter *brightnessFilter = [GPUImageBrightnessFilter new];
    if (properties.brightness != 0.0)
    {
        brightnessFilter.brightness = properties.brightness;
        [filtersList addObject:brightnessFilter];
    }

    GPUImageContrastFilter *contrastFilter = [GPUImageContrastFilter new];
    if (properties.contrast != 1.0)
    {
        contrastFilter.contrast = properties.contrast;
        [filtersList addObject:contrastFilter];
    }

    GPUImageSaturationFilter *saturationFilter = [GPUImageSaturationFilter new];
    if (properties.saturation != 1.0)
    {
        saturationFilter.saturation = properties.saturation;
        [filtersList addObject:saturationFilter];
    }

    if (filtersList.count > 0)
    {
        [source addTarget:filtersList.firstObject];
        for (int i = 0; i < filtersList.count - 1; i++)
        {
            [(id)[filtersList objectAtIndex:i] addTarget:[filtersList objectAtIndex:i + 1]];
        }

        [source addTarget:filtersList.lastObject];

        [(id)filtersList.lastObject useNextFrameForImageCapture];

        [source processImage];

        return [filtersList.lastObject imageFromCurrentFramebuffer];
    }
    else
        return self;
}

@end

Answer 1

To expand upon my comment, the problem here appears to be the line

[source addTarget:filtersList.lastObject];

For the rest of your logic, you're stepping through and attaching filters properly by targeting them to the next one in line. However, that one line at the end messes up the ordering by attaching the source to the last filter in the chain directly.

If that last filter only takes on input, that input will be overridden by your source. This will result in a dangling filter chain that is forked by the dual directions of frames after your source (one to the now incomplete filter chain, one directly to the last filter). I may not account for this properly, thus the overreleasing of the framebuffer I warn about in that assertion.

If I might make one last comment, there is some overhead in creating a GPUImagePicture based on a UIImage, so if you can find a way to not have to create that every time you needed to apply a filter to your image your overall processing will be much faster. Similarly, if you can keep the filters around that will be reused, you can save a little on initial filter setup. That may not work in your architecture, but it is something to be aware of.