Search code examples
scalaapache-sparktwitter4jspark-streaming

Convert DStream[java.util.Date] to DStream[String]

I'm trying to union two Dstreams:

val statuses = tweets.map(status => status.getText())
    val users = tweets.map(status => status.getUser())
    val Dates = tweets.map(status => status.getCreatedAt())
    (statuses. union(Dates)).print

But I'm getting an error that there is a mismatch in types:

Found: org.apache.spark.streaming.dstream.DStream[java.util.Date]

Required: org.apache.spark.streaming.dstream.DStream[String]

How can I do the conversion?

Solution

  • try this

    val Dates = tweets.map(status => status.getCreatedAt.toString)

    or if you want specific format

    val format = new SimpleDateFormat("yyyy-MM-dd")
val Dates = tweets.map(status => format.format(status.getCreatedAt))