Add number of days to date field in spark scala Velocity

Question

Can anyone tell how to add days to already existing days.Example: I want to add 30 days to start date and i want to get it printed. I am trying this

set( $pageParameters.wiEndDate = ($pageParameters.wiStartDate+ 30))

But it is not working. Can anyone tell is there any API or how to get? Thank you

Answer 1

If you are looking to add a days in a DateType field you can use a function org.apache.spark.sql.functions._date_add(column, no of days)

You can add a month if you want in a existing DateType field with

org.apache.spark.sql.functions._add_months()

Here is simple example to make you clear.

  val spark =
    SparkSession.builder().master("local").appName("test").getOrCreate()

  import spark.implicits._

  val data = spark.sparkContext.parallelize(
    Seq((0, "2016-01-1"),
        (1, "2016-02-2"),
        (2, "2016-03-22"),
        (3, "2016-04-25"),
        (4, "2016-05-21"),
        (5, "2016-06-1"),
        (6, "2016-03-21"))
  ).toDF("id", "date")

  //Cast to date type
  val data1 = data.withColumn("date", $"date".cast(DateType))

  //add 1 month in each row 
  data.withColumn("date", add_months($"date", 1)).show

  //add 30 days in each row
  data.withColumn("date", date_add($"date", 30)).show