I have created an external table using Glue catalog and trying to read a CSV file from S3. However, the three timestamp fields in my CSV file are all showing as NULL while the other values are shown as proper values.
I checked the serialization parameter and it was {"serialization.format":","}. Even when i removed the serialization parameter from the AWS Glue console from the table, it was not reading the timestamp values.
CREATE EXTERNAL TABLE spectrum_schema.ext_mv_party_edu1
(
col1 varchar(56),
col2 timestamp,
col3 timestamp,
col4 VARCHAR(4),
.
.
.
coln timestamp
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
STORED AS textfile
LOCATION 's3://bucketname/folder1/folder2/folder3/'
TABLE PROPERTIES ('skip.header.line.count'='1');
The timestamp format in the file was different than the one mentioned in documentation. Hence the null value. Marking this as closed.