Search code examples
jsonvalidationavroapache-nifi

Apache Nifi Validating Nested Json

I am using the Apache Nifi validate record using Avro schemas against a nested Json. I am having particular problems with map fields in the JSON.

Schema:

{
  "type": "record",
  "name": "Test",
  "namespace": "test_events",
  "fields": [
    {
      "name": "id",
      "type": "string"
    },
    {
      "name": "test_attributes",
      "type": {
        "type": "record",
        "name": "test_attributes.instance",
        "fields": [
          {
            "name": "normal_field",
            "type": [
              "null",
              "string"
            ]
          },
          {
            "name": "nested_field",
            "type": [
              "null",
              {
                "type": "map",
                "values": [
                  "null",
                  "int"
                ]
              }
            ]
          }
        ]
      }
    }
  ]
}

Json data:

{
  "id": "aplhanum123",
  "test_attributes": {
    "normal_field": "normal",
    "nested_field": {
      "meaning": 42
    }
  }
}

Invalidation message:

Records in this FlowFile were invalid for the following reasons: ; The following 1 fields had values whose type did not match the schema: [/test_attributes/nested_field]

I do not understand this message. I'd imagine that nested_field in the Json is valid map and therefor should not be invalidated by the processor. What am i missing?

Edit: * Removed the } in sample JSON. * Nifi-1.7.1, ValidateRecord with JsonTreeReader

Solution

  • Found NIFI-5678 (acknowledging the presence of the bug) with https://github.com/apache/nifi/pull/3060/commits/c66de368d07f37a8652c09e4b116b232972821ac which has been released in version 1.8.0.