Search code examples
apache-kafkaschemaavroconfluent-platformconfluent-schema-registry

Avro schema issue - throwing null pointer exception but data is valid


Here is an example of my data that is being sent but for some reason i am getting a null pointer exception, i tried to strip some of the code out to make it smaller but think maybe i need to keep a lot of the schema to show format and what is nested fields. Have edited to only show relevant parts as do not wish to advertise my schema online

{
    "eventType": "XXX",
    "correlationId": "XXX",
    "timestamp": XXX,
    "policy": {
        "policyNumber": "XXX",
        "policyId": "XXX",
        "customerNumber": "XXX",
        ......(lots more fields like this)
        "insuredsAndDrivers": [{
            ......(lots more fields )
            "driver": {
                "relationshipToInsuredCd": "IN",
                "driverTypeCd": "P",
                "occupationCd": null,
                "driverLicenses": [{
                    "licenseTypeCd": null,
                    "licenseDate": null,
                    "licenseStateProvCd": null,
                    "licensePermitNumber": null
                }]
            }
        }, {
            ......(lots more fields like this)
            },
            "driver": null
        }],
        "vehicles": [{
            .....(lots of single fields here)
            "coverages": {
                .....(lots of single fields here)
            }
        }],
        "customer": {
            "extensionFields": {
                "nif": "XXX"
            },
            "individualDetails": {
                ......(lots more fields like this)
            },
            "phones": [{
                "phoneNumber": "1234567890"
            }],
            "addresses": [{
                ......(lots more fields like this)
            }]
        },
        "previous_policy": null
    }
}

Here is the Avro schema sorry its quite long

{
    "type": "record",
    "namespace": "my.namespace",
    "name": "MyEvent",
    "fields": [
                    ....lots of fields here
                    {
                        "name": "insuredsAndDrivers",
                        "default": [],
                        "type": {
                            "type": "array",
                            "items": {
                                "name": "InsuredsAndDrivers_Record",
                                "default": "",
                                "type": "record",
                                "fields": [
                                     ......(lots more fields here),
                                    {
                                        "name": "driver",
                                        "type": {
                                            "name": "Driver",
                                            "default": "",
                                            "type": "record",
                                            "fields": [{
                                                    "name": "relationshipToInsuredCd",
                                                    "type": ["string", "null"],
                                                    "default": ""
                                                },
                                                {
                                                    "name": "driverTypeCd",
                                                    "type": ["string", "null"],
                                                    "default": ""
                                                },
                                                {
                                                    "name": "occupationCd",
                                                    "type": ["string", "null"],
                                                    "default": ""
                                                },
                                                {
                                                    "name": "driverLicenses",
                                                    "default": [],
                                                    "type": {
                                                        "type": "array",
                                                        "items": {
                                                            "name": "DiverLicenses_Record",
                                                            "default": "",
                                                            "type": "record",
                                                            "fields": [{
                                                                    "name": "licenseTypeCd",
                                                                    "type": ["string", "null"],
                                                                    "default": ""
                                                                },
                                                                {
                                                                    "name": "licenseDate",
                                                                    "type": ["string", "null"],
                                                                    "default": ""
                                                                },
                                                                {
                                                                    "name": "licenseStateProvCd",
                                                                    "type": ["string", "null"],
                                                                    "default": ""
                                                                },
                                                                {
                                                                    "name": "licensePermitNumber",
                                                                    "type": ["string", "null"],
                                                                    "default": ""
                                                                }
                                                            ]
                                                        }
                                                    }
                                                }
                                            ]
                                        }
                                    }
                                ]
                            }
                        }
                    }

               ...rest of schema not relevant

I am getting the error Caused by: java.lang.NullPointerException: null of com.lm.gde.eventing.avro.Driver of com.lm.gde.eventing.avro.InsuredsAndDrivers_Record of array of com.lm.gde.eventing.avro.Policy of com.lm.gde.eventing.avro.EnrichedPolicyEvent


Solution

  • your 2nd "InsuredsAndDrivers_Record" element of the array "insuredsAndDrivers":

    {
       "firstName": "XXX",
       "middleName": "Driver secondLastName",
       ...... (lots more fields here)
       "phones": [],
       "emails": [],
       "insured": {
           "primaryInsuredInd": "false"
       },
       "driver": null  <---- THIS
    }
    

    the driver field is null, yet the schema defines it as:

    "name": "insuredsAndDrivers",
    "default": [],
    "type": {
        "type": "array",
        "items": { ... }
    }
    

    which is not nullable. you need to either supply driver or make the item type of the array a union with null.