Search code examples
iosswiftfirebase-mlkit

Is there a way to take highest confidence result only in VisionLabelDetector?

To archive this, I was thinking about take 1 result only which is the top one so I check the documentation and VisionCloudDetectorOptions have this variable maxResults so if I set it to 1, my goal is complete but this only work with Cloud-based image labeling. So I check VisionLabelDetectorOptions which run locally and there is no option for that. And the result when printing out from this

return "Label: \(String(describing: feature.label)), " +
      "Confidence: \(feature.confidence), " +
      "EntityID: \(String(describing: feature.entityID)), " +
      "Frame: \(feature.frame)"
    }.joined(separator: "\n")

Will turn into this

Label: Food, Confidence: 0.795696, EntityID: /m/02wbm, Frame: (0.0, 0.0, 0.0, 0.0)
Label: Fruit, Confidence: 0.71232, EntityID: /m/02xwb, Frame: (0.0, 0.0, 0.0, 0.0)
Label: Vegetable, Confidence: 0.595484, EntityID: /m/0f4s2w, Frame: (0.0, 0.0, 0.0, 0.0)
Label: Plant, Confidence: 0.536178, EntityID: /m/05s2s, Frame: (0.0, 0.0, 0.0, 0.0)

These are sample code I run from Firebase/quickstart-ios starting at line 645.

My second solution is to do the topResult like in CoreML which is using VNClassificationObservation to return the first result. Something like this

guard let results = request.results as? [VNClassificationObservation],
            let topResult = results.first else {
                fatalError("Unexpected result")
        }

but I haven't figured out how to replicate like that yet.

So, is there a way to take the highest Confidence label only? In this case is the Food label.

Solution

  • Assuming labels is the array containing all the VisionLabel objects returned by VisionLabelDetector.detect(in:completion:), usually all the labels in the array are already sorted based on their confidence from highest to lowest, so you can simply get the highest confidence label with labels.first.

    If you want to be extra safe and pick the highest confidence label yourself, you could do the following:

    let topLabel = labels.max(by: { (a, b) -> Bool in return a.confidence < b.confidence })