How to skip file headers in impala external table?

Question

I have file on HDFS with 78 GB size

I need to create an Impala External table over it to perform some grouping and aggregation on data available

Problem The file contain headers.

Question Is there any way to skip headers from file while reading the file and do querying on the rest of data.

Although i have a way to solve the problem by copying file to local then remove the headers and then copy the updated file to HDFS again but that is not feasible as the file size is too large

Please suggest if anyone have any idea...

Any suggestions will be appreciated....

Thanks in advance

Answer 1

UPDATE or DELETE row operations are not available in Hive/Impala. So you should simulate DELETE as

• Load data file into a temporary Hive/Impala table
• Use INSERT INTO or CREATE TABLE AS on temp table to create require table