VNClassificationObservation not working?

Question

I have been hard at work trying to create a simple computer vision application in swift using CoreML and Vision. I have trained my own Keras network that takes colored images with 64x64 resolution and then tells you which letter of the alphabet it is. When I execute this App on my phone and take an image and hit 'use this image', the code crashes at this piece of code:

//send a request to the network to identify the image
            let request = VNCoreMLRequest(model: model) { (request, error) in
                guard let results = request.results as? [VNClassificationObservation] else {
                    fatalError("Model failed to load image")
            }

I have gotten stuck on this error for the past three hours and hope you guys could help me figure out what is wrong! Below is the rest of the code I used.

class ViewController: UIViewController, UIImagePickerControllerDelegate, UINavigationControllerDelegate {

    @IBOutlet weak var imageView: UIImageView!

    let imagePicker = UIImagePickerController()

    override func viewDidLoad() {
        super.viewDidLoad()

        imagePicker.delegate = self
        imagePicker.sourceType = .camera
        imagePicker.allowsEditing = false

    }

    //function to chose an image from your library or take one with your camera
    func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {

        if let userPickedImage = info[UIImagePickerControllerOriginalImage] as? UIImage {

            imageView.image = userPickedImage

            //transform the image from UIImage to a CIIMage type
            guard let ciImage = CIImage(image: userPickedImage) else {
                fatalError("Couldn't transform image to CIImage type")
            }

            //call the detect function to pass the new ciImage into the network
            detect(image: ciImage)

        }

        imagePicker.dismiss(animated: true, completion: nil)

    }

    //function to classify the image that is taken with the camera or chosen from the library
    func detect(image: CIImage) {

        //try to load the model, if not throw an error
        guard let model = try? VNCoreMLModel(for: chars74k().model) else {
            fatalError("Loading coreML model failed")
        }

        //send a request to the network to identify the image
        let request = VNCoreMLRequest(model: model) { (request, error) in
            guard let results = request.results as? [VNClassificationObservation] else {
                fatalError("Model failed to load image")
        }

            print(results)

    }
        //create handler for image
        let handler = VNImageRequestHandler(ciImage: image)

        do{
            try handler.perform([request])
        }
        catch {
            print(error)
        }

Answer 1

You are casting on the wrong type of your results. To check the type of your results you can put a breakpoint in the fatalError line and do it with debugger. But I assume you are a beginner so try this. Replace:

guard let results = request.results as? [VNClassificationObservation] else {
    fatalError("Model failed to load image")
}

with:

print("your results are \(type(of: results))")

if let results = request.results as? [VNClassificationObservation] {
    print("your results are of type VNClassificationObservation")
}

if let results = request.results as? [VNPixelBufferObservation] {
    print("your results are of type VNPixelBufferObservation")
}

if let results = request.results as? [VNCoreMLFeatureValueObservation] {
    print("your results are of type VNCoreMLFeatureValueObservation")
}

The docs are pretty clear on possible outcome of VNCoreMLRequest so you should read them (I highly recommend this even if you succeed without that, it's not much and they are simple).

Most of the cases output of your request would be something combining classification with other outputs of the model, so chances are that your failing closure should be replaced with:

guard let results = request.results as? [VNCoreMLFeatureValueObservation] else {
    fatalError("Model failed to load results")
}

remember to modify the code which refers to results accordingly. VNCoreMLFeatureValueObservation protocol contains information about possible values. You can also print them to the debugger from your model with:

print(model.modelDescription.outputDescriptionsByName)

This should shed some light on what you should expect.