Search code examples
scalaapache-sparksprayapache-zeppelinspray-json

Spray-JSON in an Apache Zeppelin Notebook


I am using Zeppelin notebooks to write a prototype for a Spark Streaming application. It receives small JSON messages via an event bus and I need to parse these in a (preferably) distributed manner. I chose spray-json to deserialize the individual messages, but I cannot seem to be able to get it working. I think believe that this is because Zeppelin notebooks are interpreted via some sort of REPL interface.

I directly copied this sample from the docs:

case class Color(name: String, red: Int, green: Int, blue: Int)
object Color

object MyJsonProtocol extends DefaultJsonProtocol {
  implicit val colorFormat = jsonFormat4(Color.apply)
}

But it gives me the following output:

defined class Color
defined object Color
warning: previously defined class Color is not a companion to object Color.
Companions must be defined together; you may wish to use :paste mode for this.
<console>:75: error: value apply is not a member of object Color
 Note: implicit value colorFormat is not applicable here because it comes after the application point and it lacks an explicit result type
         implicit val colorFormat = jsonFormat4(Color.apply)

Is there another way that does allow me to deserialize my messages in a Zeppelin notebook? I am not bound to spray-json, but it does seem like a nice library.


Solution

  • Thanks to @philantrovert's comment, I was able to get it to work. The trick is to put the class and object declarations on the same line like this:

    case class Color(name: String, red: Int, green: Int, blue: Int); object Color;