Search code examples
data-warehousepolybase

How to change extension and compression way?


I'm following the steps here.

Q1: I success in exporting CSV. But the extension is TXT. Is it possible to change it to CSV?

Q2: There are two compression methods provided, Gzip and Deflate. I've tried both. The extensions are gz and deflate. Is it possible to compress to a zip file? Is it possible to export multiple files as a zip file?


Solution

  • It is not possible to change the file extension. You are stuck with .txt. It is not possible to export to zip file. You are stuck with default (no compression) or gzip (gz). It is not possible (at least via Polybase) to export multiple files to a single zip file. This is a bad practice for Polybase / external files anyway as the import can not parallelise over multiple files in a single zip file during import.

    Do you need to change these? Most ETL / ELT tools like SSIS, Azure Data Factory, Talend, Informatica etc will be able to cope with this.

    Polybase is a powerful tool but with a fairly simple interface. You don't have that much control over the settings, other than those documented. For more fine-grain control, eg choice of filenames, compression type etc, consider Azure Data Factory which can handle this type of thing.