jackson databind Json serialization is writing Some(99): Option[Int] as {"empty":true,"defined":false}

Question

I'm using Jackson in my Scala/Spark program and I've distilled my issue to the simple example below. My problem is that when my case class has the Option[Int] field (age) set to None I see reasonable deserialization output (that is: a struct with empty=true). However, when age is defined, i.e., set to some Int like Some(99), I never see the integer value in the deserialization output .

Given :

import com.fasterxml.jackson.databind.ObjectMapper
import java.io.ByteArrayOutputStream
import scala.beans.BeanProperty

case class Dog(@BeanProperty name: String, @BeanProperty age: Option[Integer])

object OtherTest extends App {
  jsonOut(Dog("rex", None))
  jsonOut(Dog("mex", Some(99)))


  private def jsonOut(dog: Dog) = {
    val mapper = new ObjectMapper();
    val stream = new ByteArrayOutputStream();

    mapper.writeValue(stream, dog);
    System.out.println("result:" + stream.toString());
  }
}

My output is as shown below. Any hints/help greatly appreciated !

result:{"name":"rex","age":{"empty":true,"defined":false}}
result:{"name":"mex","age":{"empty":false,"defined":true}}

Update after the Helpful Answer

Here are the dependencies that worked for me:

    implementation 'org.scala-lang:scala-library:2.12.2'

    implementation  "org.apache.spark:spark-sql_2.12:3.1.2"
    implementation "org.apache.spark:spark-sql-kafka-0-10_2.12:3.1.2"
    implementation "org.apache.spark:spark-avro_2.12:3.1.2"
    implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.12:2.10.0'

Here is the updated code (with frequent flyer bonus - round trip example):

  private def jsonOut(dog: Dog) = {
    val mapper = new ObjectMapper()
    mapper.registerModule(DefaultScalaModule)
    val stream = new ByteArrayOutputStream();
    mapper.writeValue(stream, dog);
    val serialized = stream.toString()
    System.out.println("result:" + serialized);
    // verify we can read the serialized thing back to case class:
    val recovered = mapper.readValue(serialized, classOf[Dog])
    System.out.println("here is what we read back:" + recovered);
  }

Here is the resultant output (as expected now ;^) ->

> Task :OtherTest.main()
result:{"name":"rex","age":null}
here is what we read back:Dog(rex,None)
result:{"name":"mex","age":99}
here is what we read back:Dog(mex,Some(99))

Answer 1

You need to add the Jackson module for Scala to make it work with standard Scala data types.

1. Add this module as your dependency: https://github.com/FasterXML/jackson-module-scala
2. Follow the readme on how to initialize your ObjectMapper with this module.