Search code examples
c#hololensazure-cognitive-servicesface-api

face API Exception: Unable to deserialize the response


I am working on an app that performs face recognition on the Hololens 2 using Unity 2021.3.4f1. I'm having trouble running the app on the Hololens while everything works fine in the editor. I'm using the Microsoft Azure Cognitive Services face API Version 2.8.0.0 as described here or here.

This is a snippet from the code I use:

private async Task DetectFaceExtractAsync(IFaceClient client, string filepath, string recognitionModel)
{
    Debug.Log("--- DETECT FACE ---");

    IList<DetectedFace> detectedFaces;

    // Specify facial features to be retrieved
    List<FaceAttributeType> features = new List<FaceAttributeType>
    {
         FaceAttributeType.Blur, FaceAttributeType.Emotion, FaceAttributeType.Exposure,  FaceAttributeType.HeadPose,
         FaceAttributeType.Noise, FaceAttributeType.Occlusion, FaceAttributeType.Smile, FaceAttributeType.QualityForRecognition
    };

    // Detect faces with all attributes from image.
    try
    {
        using (FileStream faceimagestream = File.OpenRead(filepath))
        {
            detectedFaces = await client.Face.DetectWithStreamAsync(
                faceimagestream, 
                returnFaceAttributes: features, 
                detectionModel: DetectionModel.Detection01, 
                recognitionModel: recognitionModel);
        }
    }
    catch (Exception e)
    {
        Debug.Log("Exception: " + e.Message);
        throw e;
    }}

When receiving a response from Azure I get an "Unable to deserialize the response" Exception. For some reason, this exception does not occur in the editor and only occurs when a face should be recognizable from the image. If there is no face in the image, the response is apparently deserializable.

This is the debug output: enter image description here

Update:

The exception also occurs if I ask for the following features, which should still be available with the new limitations from the 21.06.22:

List<FaceAttributeType> features = new List<FaceAttributeType>
    {
         FaceAttributeType.Blur, FaceAttributeType.Exposure,  FaceAttributeType.HeadPose,
         FaceAttributeType.Noise, FaceAttributeType.Occlusion, FaceAttributeType.QualityForRecognition
    };

Solution

  • To perform the deserialization method in the current version, it is suggestable to convert the response into JSON and then perform deserialization. As mentioned in the official documentation, deserialization in Face API is suggestable to perform with JSON. The Hololens 2, is missing concept, where there are no official procedures to handle the errors occurring with version 2.