Search code examples
javaobject-detectiongoogle-mlkit

Display Label with Minimum Threshold


I'm modifying a custom object detector based off this sample provided by google: vision-quickstart I'd like to prevent displaying a detected object altogether if that object doesn't meet a specified confidence level.

Original server part object detection

Then I modified the code to delete the label text with the following:

// ObjectGraphic.java
private static final float CONFIDENCE_THRESHOLD = 0.50f;
...
for (Label label : object.getLabels()) {
  if (label.getConfidence() < CONFIDENCE_THRESHOLD) {continue;}
  canvas.drawText...

Label Text Removed but not the bounding/label boxes enter image description here Obviously, this is one step in the right direction but not the final objective. Here's what I found so far:

// ObjectDetectorProcessor.java

  @Override
  protected void onSuccess(
      @NonNull List<DetectedObject> results, @NonNull GraphicOverlay graphicOverlay) {
    for (DetectedObject object : results) {
      graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));
      // commenting this preventing object overlays from being drawn
      // possibly add graphicOverlay.remove() based on label.confidence, but how do I do that?
    }
  }
// GraphicOverlay.java
  /** Adds a graphic to the overlay. */
  public void add(Graphic graphic) {
    synchronized (lock) {
      graphics.add(graphic);
    }
  }

  /** Removes a graphic from the overlay. */
  public void remove(Graphic graphic) {
    synchronized (lock) {
      graphics.remove(graphic);
    }
    postInvalidate();
  }

Please let me know if I'm not mentioning anything. I'd be more than willing to provide additional information for your questions.


Solution

  • Solution

    Figured it out and no changes are required for any other files. Turned out to be quite simple but I'm just not familiar with java. Not sure if nesting for loops is the optimal solution but it works :/

    // ObjectDetectorProcessor.java
    import com.google.mlkit.vision.objects.DetectedObject.Label;
    ...
    private static final float CONFIDENCE_THRESHOLD = 0.5f;
    ...
    // Before (right image)
    @Override
    protected void onSuccess(
        @NonNull List<DetectedObject> results, @NonNull GraphicOverlay graphicOverlay) {
      for (DetectedObject object : results) {
        graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));
      }
    }
    // After (left image)
    protected void onSuccess(
        @NonNull List<DetectedObject> results, @NonNull GraphicOverlay graphicOverlay) {
      for (DetectedObject object : results) {
        for (Label label : object.getLabels()) {
          if (label.getConfidence() < CONFIDENCE_THRESHOLD) {continue;}
          else {graphicOverlay.add(new ObjectGraphic(graphicOverlay, object));}
        }
      }
    }
    

    Before (right) and After (left) Additionally, I found that the above results (removing just the label text, but not the boxes) can be reproduced with the following built-in method. Found in PreferenceUtils.java of the mlkit vision-quickstart google sample. Direct link to method below: https://developers.google.com/android/reference/com/google/mlkit/vision/objects/custom/CustomObjectDetectorOptions.Builder#public-customobjectdetectoroptions.builder-setclassificationconfidencethreshold-float-classificationconfidencethreshold