How to pass an extra parameter to Vision framework?

Question

I have a Swift / CoreML code that works fine. I want to simplify my code by using the Vision framework. In this code, there is a UIImage -> CGImage -> CVPixelBuffer conversion that I would like to get rid of. I know that using Vision one can pass directly a CGImage as input parameter. The problem I have is that my model takes 2 inputs (image + MLMultiArray) and output and image:

Inputs

    my_input : Image (Color 512 x 512)
    my_sigma : MultiArray (Float32 1)

Outputs

    my_output : Image (Color 512 x 512)

I've tried to pass the sigma parameter as follow:

guard let cgImage = uiImage.cgImage else {
    return nil
}

let options:[VNImageOption: Any] = [VNImageOption(rawValue: "my_sigma"): 0.1]

let handler = VNImageRequestHandler(cgImage: cgImage, options: options)
do {
    try handler.perform(visionRequest)
} catch {
    print(error)
}

By doing so, I get the following error:

[coreml] Failure verifying inputs. no results:Error Domain=com.apple.vis Code=3 "The VNCoreMLTransform request failed" UserInfo={NSLocalizedDescription=The VNCoreMLTransform request failed, NSUnderlyingError=0x280cbbab0 {Error Domain=com.apple.CoreML Code=0 "Required input feature not passed to neural network."

So, it seems that I didn't pass correctly the second parameter to the request handler. I have not been able to find the answer.

• What is the correct way to pass such a parameter to Vision?
• Do you know if I can use Vision to output an image directly?

Thank you for you help.

Answer 1

Have a look at the featureProvider property of VNCoreMLRequest. This is an object that provides additional inputs, i.e. anything other than the image input.