Search code examples
firebaseface-detectionfirebase-mlkitgoogle-mlkitfirebase-machine-learning

MLKit rotate face image so its straight (iOS and Android)

I'm developing an app where I use Firebase's MLKit to do face detection. I then take the detected face and run it through a custom tensorflow model

The problem is my custom tensorflow model expects the faces to be straight on. If a user's face is rotated at all, I'm not getting accurate results

Is there any way to rotate a face image so its straight? How would I do that with MLKit?

My current way of getting the face image is using AVCaptureSession (iOS) and cutting out the image based on the bounds given to me in the face object from mlkit. It works but again if the user's face isn't straight on, I end up with a rotated face

Solution

  • You could use this head rotation degree to rotate the head back. But it may only work with EulerAngleZ() where the head rotation is in XY plane.