Search code examples
jsonavro

Avro: deserialize json - schema with optional fields

There are a lot of questions and answers on stackoverflow on the subject, but no one that helps.

I have a schema with optional value:

{
 "type" : "record",
 "name" : "UserSessionEvent",
 "namespace" : "events",
 "fields" : [ {
   "name" : "username",
   "type" : "string"
 }, {
   "name" : "errorData",
   "type" : [ "null", "string" ],
   "default" : null
 }]
}

And I'm trying deserialize json w/o this field:

{
 "username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
 "errorData" : { "string":"070226AC-9B91-47CE-85FE-15AA17972298"}
}

using code:

val reader = new GenericDatumReader[GenericRecord](schema)
val decoder = DecoderFactory.get().jsonDecoder(schema, json)
reader.read(null, decoder)

and I got: org.apache.avro.AvroTypeException: Expected field name not found: errorData

The only way that works is json

{
 "username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
 "errorData" : null

}

Is there a way to deserialize json w/o this field?

Another question: when this field is here, I should write

{
 "username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
 "errorData" : { "string":"070226AC-9B91-47CE-85FE-15AA17972298"}

}

Is there a way to deserialize a "normal" json:

{
 "username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
 "errorData" : "070226AC-9B91-47CE-85FE-15AA17972298"
}

?

Solution

  • case 1 is working fine in java . 

    {
"username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
"errorData" : { "string":"070226AC-9B91-47CE-85FE-15AA17972298"}
}

    for case 2 Your schema is defined for union. You can update you schema as below to deserialize json.

     {
   "username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
   "errorData" : "070226AC-9B91-47CE-85FE-15AA17972298"
 }

{
  "type" : "record",
  "name" : "UserSessionEvent",
  "namespace" : "events",
  "fields" : [ {
                "name" : "username",
                "type" : "string"
             }, {
                "name" : "errorData",
                "type" :  "string" ,
                "default" : null
               }]
}