Search code examples
apache-pig

Trying to covert long to ToDate format

My input is long "20190503143744" and wanted to convert to format "2019-09-06 11:46:22"

Trying below code:

A = LOAD 'stp_master_subscriber_profile' using org.apache.hive.hcatalog.pig.HCatLoader() as (mdn:chararray, imei:chararray, imsi:chararray, subscriber_id:long, manufacturer:chararray, model:chararray, update_time:long, scenario:chararray, vz_customer:chararray, commit_time:long);
B = FOREACH A GENERATE ToString(ToDate((chararray)commit_time,'yyyyMMdd'),'yyyy-MM-dd HH:mm:ss') as event_date_gmt:chararray;

Getting error:

ERROR 1066: Unable to open iterator for alias B. Backend error : org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-15 Operator Key: scope-15) children: null at []]: java.lang.IllegalArgumentException: Invalid format: "20190503143744" is malformed at "143744"
Solution

  • The issue is that you're specifying the format as yyyyMMdd but your original input is in yyyyMMddHHmmss format, so you get an error when Pig reaches 143744 instead of the end of your string. Try this:

    B = FOREACH A GENERATE ToString(ToDate((chararray)commit_time,'yyyyMMddHHmmss'),
    'yyyy-MM-dd HH:mm:ss') as event_date_gmt;