from_unixtime gave me an awkward value

Question

Hi I used from_unixtime to convert this value 1632837232439 and I got 53712-07-21 01:53:59 is this right? I can't make sense of this, I used

df = df.select(from_unixtime(df_sixty60['createdOn']).alias("date_key"))

Thanks for you help even if you can suggest other ways of representing this. Thanks

Answer 1

Try with to_timestamp() function and divide by 1000 as your epoch timestamp has milliseconds included.

Example:

df.show()
#+-------------+
#|    createdOn|
#+-------------+
#|1632837232439|
#+-------------+ 

df.select(to_timestamp(df['createdOn']/1000).alias("date_key")).show(10,False)

#+-----------------------+
#|date_key               |
#+-----------------------+
#|2021-09-28 13:53:52.439|
#+-----------------------+