Search code examples
scalaapache-sparkapache-spark-sql

Date conversion to timestamp in EPOCH


I am looking to convert date to day minus 1 12:00 AM (epoch time) in spark

val dateInformat=DateTimeFormatter.ofPattern("MM-dd-yyyy")
val batchpartitiondate= LocalDate.parse("10-14-2022",dateInformat)
batchpartitiondate: javatimelocalDate=2022-10-14

batchpartitiondate should be converted to  epochtime(1665619200)

Date for example:

InputDate in spark submit argument is 12-15-2022

I need the output as epoch time (1665705600) i.e in GMT:Friday,October 14,12:00:00AM

if i give input as 12-14-2022 it should give the output as epoch time (1665619200) i.e in GMT:Thursday,October 13,12:00:00AM


Solution

  • Does this achieve what you are looking to do?

        val dateInFormat = DateTimeFormatter.ofPattern("MM-dd-yyyy")
        val batchPartitionDate = LocalDate.parse("10-14-2022", dateInformat)
        val alteredDateTime = batchPartitionDate.minusDays(1).atStartOfDay()
    
        // current zone
        {
          val zone = ZoneId.systemDefault()
          val instant = alteredDateTime.atZone(zone).toInstant
          val epochMillis = instant.toEpochMilli
        }
        
        // UTC
        // Or you can specify the appropriate timezone insteasd of UTC
        {
          val zone = ZoneOffset.UTC
          val instant = alteredDateTime.toInstant(zone)
          val epochMillis = instant.toEpochMilli
        }