Search code examples
delta-lake

can we vacuum _delta_log folder in delta lake


We use vacuum to delete older data files in delta lake api.

  1. does vacuum also delete files from _delta_log ?
  2. if no, is there any way we vacuum _delta_log

Solution

  • From doc:

    vacuum deletes only data files, not log files. Log files are deleted automatically and asynchronously after checkpoint operations. The default retention period of log files is 30 days, configurable through the delta.logRetentionDuration property which you set with the ALTER TABLE SET TBLPROPERTIES SQL method.

    So you don't have to run anything to clean the _delta_log, old files are removed from it as new versions of the table are created.