Search code examples
gzipgoogle-bigquerygoogle-cloud-storageavro

BigQuery load compressed data from Cloud Storage

I have a lot of *.gz files in my Google Cloud Storage. I want to load those data to BigQuery. I've tried to execute 

bq load --source_format=AVRO projectId:dataset.table gs://bucket/*.gz

But received error

The Apache Avro library failed to parse file gs://bucket/f92d8ae3-6eba-4e35-9fc0-b8f31b4b9881-part-r-00004.gz.

Is it possible to upload compressed data to BigQuery? What is the best pracrise for those problems?

Solution

  • Compressed Avro files are not supported ... See more in Avro format