AWS Redshift has recently released their own new encoding format AZ64, for which they say:
Compared to ZSTD encoding, AZ64 consumed 5–10% less storage, and was 70% faster
When I use an ANALYZE COMPRESSION my_table I still receive ZSTD as an encoding format for all of its columns.
So is it really recommended as an encoding format over ZSTD? Shall I naively prefer AZ64 whenever possible to use it?
I got a reply from AWS Support on this question:
About your question preferring AZ64 instead of ZSTD were possible, yes you can do it.
Given AZ64 provide better performance compared to ZSTD
For further exaplanation:
Yes, AZ64 is better than ZSTD. It has comparable compression, when compared with ZSTD but greatly better performance, which is something you would have already learned using. As of now
ANALYZE COMPRESSIONcommand doesn't support AZ64, also I don't have an ETA on when AZ64 will be available withANALYZE COMPRESSION. I will suggest you to keep an eye on
- https://docs.aws.amazon.com/redshift/latest/mgmt/rs-mgmt-cluster-version-notes.html
- https://aws.amazon.com/redshift/whats-new/
for any updates on AWS Redshift. I have verified this with the internal service team.
ANALYZE COMPRESSIONis an advisory tool it recommends the optimal column encoding depending upon the columns.