How to get corners using GPUImageHarrisCornerDetectionFilter

Question

I am trying to get the corner points from a still image using GPUImageHarrisCornerDetectionFilter.

I have looked at the example code from the project, I have looked at the documentation, and I have looked at this post that is about the same thing: GPUImage Harris Corner Detection on an existing UIImage gives a black screen output

But I can't make it work - and I have a hard time understanding how this is supposed to work with still images.

What I have at this point is this:

func harrisCorners() -> [CGPoint] {
    var points = [CGPoint]()

    let stillImageSource: GPUImagePicture = GPUImagePicture(image: self.image)
    let filter = GPUImageHarrisCornerDetectionFilter()

    filter.cornersDetectedBlock = { (cornerArray:UnsafeMutablePointer<GLfloat>, cornersDetected:UInt, frameTime:CMTime) in
        for index in 0..<Int(cornersDetected) {
            points.append(CGPoint(x:CGFloat(cornerArray[index * 2]), y:CGFloat(cornerArray[(index * 2) + 1])))
        }
    }

    filter.forceProcessingAtSize(self.image.size)
    stillImageSource.addTarget(filter)
    stillImageSource.processImage()

    return points
}

This function always returns [] so it's obviously not working.

An interesting detail - I compiled the FilterShowcaseSwift project from GPUImage examples, and the filter fails to find very clear corners, like on a sheet of paper on a black background.

Answer 1

filter.cornersDetectedBlock = { (cornerArray:UnsafeMutablePointer<GLfloat>, cornersDetected:UInt, frameTime:CMTime) in
    for index in 0..<Int(cornersDetected) {
        points.append(CGPoint(x:CGFloat(cornerArray[index * 2]), y:CGFloat(cornerArray[(index * 2) + 1])))
    }
}

This code you have here sets a block that gets called every frame.

This is an asynchronous process so when your function returns that has never been called yet and your array should always be empty. It should be called after the frame has finished processing.

To verify this, set a breakpoint inside that block and see if it gets called.

Warning from Brad Larson (creator of GPUImage) in the comments:

The GPUImage you create here stillImageSource will be deallocated after this function exits and may cause crashes in this case.