Search code examples
sqlpysparkdatabricksaws-databricks

timestamp conversion in databricks using date_format

I would like to convert below timestamp in databricks, Please help to get desired format

select date_format(from_utc_timestamp(current_timestamp,'America/Los_Angeles'), 'MM/DD/YY HH24:MI') AS START_TIME

Error:

IllegalArgumentException: All week-based patterns are unsupported since Spark 3.0, detected: Y,
Solution

  • Error message you got:

    IllegalArgumentException: ......, detected: Y,

    As described in date_format() docs:

    ... for instance dd.MM.yyyy and could return a string like ‘18.03.1993’. All pattern letters of datetime pattern can be used.

    datetime pattern lists valid symbols. Y is not a valid symbol. Use y instead.

    It also says:

    Year: The count of letters determines the minimum field width below which padding is used. If the count of letters is two, then a reduced two digit form is used. For printing, this outputs the rightmost two digits....

    So yyyy will give you 2022 and yy will give you 22.