Search code examples
apache-sparkpyspark

Spark getting the current date from another country


I need to get the date and time of another country:

dateFormat = "%Y%m%d_%H%M"
ts=spark.sql(""" select current_timestamp() as ctime """).collect()[0]["ctime"]
ts.strftime(dateFormat)

Solution

  • You don't need pyspark for such a task, especially when you call .collect():

    import pytz
    from datetime import datetime
    
    tz = pytz.timezone('Asia/Shanghai')
    ts = datetime.now(tz)
    ts.strftime('%Y%m%d_%H%M')