Search code examples
azureazure-cognitive-servicesazure-form-recognizer

Form Recognizer: Multiple documents in one scan does not work with a labeled Model


I built a custom model with labels to read a form. This works well if the scan contains exactly one instance of the form. But as soon as the scan contains several forms of the same type, a mishmash is created. (Azure Form Recognizer v2.1-preview3 with sample labeling tool)

I would have expected that in this case several array elements would be returned in the resulting JSON under analyzeResults / documentResults (one element per detected instance of the model). But that doesn't seem to be the case. Instead, the fields are read partly from the first instance and partly from the second instance. The result is completely wrong.

Question: Do I have to divide the scan into individual documents in a preprocessing step? Wouldn't that be a predestined task for the form recognizer, since it knows the models.

Actual result:

{
    "status": "succeeded",
    "createdDateTime": "2021-04-07T09:41:33Z",
    "lastUpdatedDateTime": "2021-04-07T09:41:46Z",
    "analyzeResult": {
        "version": "2.1.0",
        "readResults": []
        "pageResults": []
        "documentResults": [
            {
                "docType": "xxxxxxx",
                "modelId": "xxxxxxx",
                "pageRange": [
                    1,
                    6
                ],
                "fields": {...}
                "docTypeConfidence": 0.778
            }
        ],
        "errors": []
    }

Expected result:

{
    "status": "succeeded",
    "createdDateTime": "2021-04-07T09:41:33Z",
    "lastUpdatedDateTime": "2021-04-07T09:41:46Z",
    "analyzeResult": {
        "version": "2.1.0",
        "readResults": []
        "pageResults": []
        "documentResults": [
            {
                "docType": "xxxxxxxx",
                "modelId": "xxxxxxxx",
                "pageRange": [
                    1,
                    3
                ],
                "fields": {...}
                "docTypeConfidence": 0.778
            },
            {
                "docType": "xxxxxxxx",
                "modelId": "xxxxxxxx",
                "pageRange": [
                    4,
                    6
                ],
                "fields": {...}
                "docTypeConfidence": 0.778
            }
        ],
        "errors": []
    }
}

Solution

  • Form Recognizer expects a document type per file, if your have several different documents or forms in one file please split the file into pages or the single documents before sending it to Form Recognizer. You can use a logic app or flow connector for this or any other simple code to split the document to pages.