How to resolve recursive decoding in Circe when parsing Json?

Question

I want to parse a JSON string using Circa. You can find an example of the input JSON below.

It's a kind of recursive data. So my property entity contains dependencies of entities.

I want to parse dependencies to map Map[String, Tasks].

{
  "entity": [
    {
      "task_id": "X",
      "type": "test",
      "attributes": {
        "name": "A",
        "random_property_count": 1 // should be ignored
      },
      "dependencies": {
        "random_name_1": {
          "entity": [
            {
              "task_id": "907544AF",
              "type": "test",
              "attributes": {
                "name": "B",
                "random_attribute": "*"
              },
              "dependencies": {
                "random_name_2": {
                  "entity": [
                    {
                      "task_id": "5",
                      "random_prop": "...",  // should be ignored as it's not present in model
                      "type": "test",
                      "attributes": {
                        "name": "C"
                      }
                    }
                  ]
                }
              }
            }
          ]
        }
      }
    }
  ]
}

Here is my code:

  case class Tasks (entity: Seq[Task])
  case class Task(task_id: String, `type`: String, attributes: Attributes, dependencies: Map[String, Tasks])
  case class Attributes(name: String)

  implicit val decodeTask: Decoder[Task] = deriveDecoder[Task]
  implicit val decodeTasks: Decoder[Tasks] = deriveDecoder[Tasks]
  implicit val decodeAttributes: Decoder[Attributes] = deriveDecoder[Attributes]

  val json = fromInputStream(getClass.getResourceAsStream("/json/example.json")).getLines.mkString
  val tasks = decode[Tasks](json)

  tasks match {
    case Left(failure) => println(failure)
    case Right(json)   => println(json)
  }

When I try to parse JSON string to my model, I get an error like this:

DecodingFailure(Attempt to decode value on failed cursor, List(DownField(dependencies), DownArray, DownField(entity), DownField(random_name_2), DownField(dependencies), DownArray, DownField(entity), DownField(random_name_1), DownField(dependencies), DownArray, DownField(entity)))

What can be the issue?

Answer 1

The second member of the DecodingFailure can be useful in cases like this, since it provides the history of successful operations that preceded the failure as well as the failing operation itself (in reverse chronological order, with the most recent first). You can print the history like this (or just inspect it in the string representation of the DecodingFailure):

scala> import io.circe.DecodingFailure
import io.circe.DecodingFailure

scala> io.circe.jawn.decode[Tasks](doc) match {
     |   case Left(DecodingFailure(_, history)) => history.reverse.foreach(println)
     | }
DownField(entity)
DownArray
DownField(dependencies)
DownField(random_name_1)
DownField(entity)
DownArray
DownField(dependencies)
DownField(random_name_2)
DownField(entity)
DownArray
DownField(dependencies)

If you follow these steps into your document up until the last one, you'll get to the following object:

{
  "task_id": "5",
  "random_prop": "...",
  "type": "test",
  "attributes": {
    "name": "C"
  }
}

The last step is the one that failed, and it's DownField(dependencies), which makes sense, given that this object doesn't have a dependencies field.

There are a couple of ways you could fix this issue. The first would be to change your JSON representation so that every object in an entity array has a dependencies field, even if it's just "dependencies": {}. If you don't want to or can't change your JSON, you could make the dependencies member an Option[Map[String, Tasks]] (I've just confirmed that this works for your case specifically). You could also define a custom Map decoder that decodes a missing field as an empty map, but that's a much more invasive approach that I wouldn't personally recommend.