Slow face detection Firebase MLKit

Question

Started playing around with MLKit face detectors with the front-facing camera, but it's really slow on processing faces

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    print("Picture at ", Date())


    let visionImage = VisionImage(buffer: sampleBuffer)
    visionImage.metadata = metadata


    faceDetector?.detect(in: visionImage) { (faces, error) in
        guard error == nil, let faces = faces, !faces.isEmpty else {
            // Error. You should also check the console for error messages.
            let errorString = error?.localizedDescription
            print("Face detection failed with error: \(errorString)")
            return
        }

  }

Where am i going wrong?

Answer 1

There are a few things you can try to speed up the detection:

1. Build your app in the release mode (optimized), as opposed to the debug mode.

2. When creating your faceDetector, please make sure to use a VisionFaceDetectorOptions and set its isTrackingEnabled to true.

3. When setting up your AVCaptureVideoDataOutput, please add the following key-value pair to its videoSettings:

key: kCVPixelBufferPixelFormatTypeKey

value: kCVPixelFormatType_32BGRA