Avro schema file references in a Avro schema

Question

I have nearly 100 avsc files and most of these avsc files refer to another asvc file usually as their type. To give an example

Item.avsc in ./com/example/common

{
    "namespace":"com.example.common",
    "name":"Item",
    "type":"record",
    "fields":[
        {
            "name":"itemId",
            "type":"com.example.common.ItemId"
        },
        {
            "name":"features",
            "type":"com.example.common.Features"
        }
    ]
}

ItemId.avsc in ./com/example/common

{
    "namespace":"com.example.common",
    "name":"ItemId",
    "type":"record",
    "fields":[
        {
            "name":"id",
            "type":"int"
        }
    ]
}

Features.avsc in ./com/example/common

{
    "namespace":"com.example.common",
    "name":"Features",
    "type":"record",
    "fields":[
        {
            "name":"Range",
            "type":{
                "type":"array",
                "items":"com.example.common.Range"
            }
        }
    ]
}

When I want to parse the schema of Item.avsc it raises:

Schema schema = new Schema.Parser().parse(new File(".\\com\\example\\common\\Item.avsc"));

Exception in thread "main" org.apache.avro.SchemaParseException: "com.example.common.ItemId" is not a defined name. The type of the "itemId" field must be a defined name or a {"type": ...} expression.

I found a workaround to this problem by using a single instance of a parser to parse the ItemId.avsc and Features.avsc first, then the Item.avsc, like below:

Parser parser = new Parser();
parser.parse(new File(".\\com\\example\\common\\ItemId.avsc"));
parser.parse(new File(".\\com\\example\\common\\Features.avsc"));
parser.parse(new File(".\\com\\example\\common\\Range.avsc"));
parser.parse(new File(".\\com\\example\\common\\Item.avsc"));

But I have nearly 100 avsc files that most of them references multiple avsc files and I need to parse each one like this while considering their dependency rank. Is there a better solution to this?

Answer 1

Iterate through your packages, and parse them over a loop.

Schema.Parser parser = new Schema.Parser();
URI uri = Test.class.getResource("package/name/here").toURI();
Path myPath = Paths.get(uri);
try (Stream<Path> paths = Files.walk(myPath)) {
     paths.filter(Files::isRegularFile)
          .filter(path -> path.toString().endsWith(".avsc"))
          .map(path -> new File(path.toUri()))
          .forEach(file -> {
              try {
                 parser.parse(file);
              } catch (IOException e) {}
          });
}