Get the cvPixelBuffer used in a VNImageRequestHandler on the VNDetectTextRectanglesRequest completion handler

Question

I am creating my request with the following code:

let textRequest = VNDetectTextRectanglesRequest(completionHandler: 
self.detectTextHandler)
textRequest.reportCharacterBoxes = true
self.requests = [textRequest]

And inside my AVCaptureVideoDataOutputSampleBufferDelegate I am creating a VNImageRequestHandler and performing it:

let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: CGImagePropertyOrientation(rawValue: 6)!, options: requestOptions)

do {
    try imageRequestHandler.perform(self.requests)
} catch {
    print(error)
}

This gives me the results of the detection inside the handler that has the following signature:

func detectTextHandler(request: VNRequest, error: Error?)

My question is, how can i get the "cvPixelBuffer" that this request used for further processing? Am I supposed to store a temporal version of it?

Answer 1

I cannot find any methods or properties to retrieve CVPixelBuffer from a VNRequest.

So, capturing it inside the closure of completionHandler would be a simple way:

In the method of AVCaptureVideoDataOutputSampleBufferDelegate:

    let pixelBuffer = ...
    let requestOptions: [VNImageOption: Any] = ...

    let textRequest = VNDetectTextRectanglesRequest {request, error in
        //### Capture `pixelBuffer` inside this closure.
        self.detectText(from: pixelBuffer, request: request, error: error)
    }
    textRequest.reportCharacterBoxes = true

    let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: CGImagePropertyOrientation(rawValue: 6)!, options: requestOptions)

    do {
        try imageRequestHandler.perform([textRequest])
    } catch {
        print(error)
    }

And use it as:

func detectText(from buffer: CVPixelBuffer, request: VNRequest, error: Error?) {
    //### Use `buffer` passed from the closure.
    //...
}