Search code examples
linuxfilefilesystemssparse-file

How does one reclaim zeroed blocks of a sparse file?


Consider a sparse file with 1s written to a portion of the file.

I want to reclaim the actual space on disk for these 1s as I no longer need that portion of the sparse file. The portion of the file containing these 1s should become a "hole" as it was before the 1s were themselves written.

To do this, I cleared the region to 0s. This does not reclaim the blocks on disk.

How do I actually make the sparse file, well, sparse again?

This question is similar to this one but there is no accepted answer for that question.

Consider the following sequence of events run on a stock Linux server:

$ cat /tmp/test.c
#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>

int main(int argc, char **argv) {
    int fd;
    char c[1024];

    memset(c,argc==1,1024);

    fd = open("test",O_CREAT|O_WRONLY,0777);
    lseek(fd,10000,SEEK_SET);
    write(fd,c,1024);
    close(fd);

    return 0;
}

$ gcc -o /tmp/test /tmp/test.c

$ /tmp/test

$ hexdump -C ./test
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002710  01 01 01 01 01 01 01 01  01 01 01 01 01 01 01 01  |................|
*
00002b10

$ du -B1 test; du -B1 --apparent-size test
4096        test
11024       test

$ /tmp/test clear

$ hexdump -C ./test
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002b10

$ du -B1 test; du -B1 --apparent-size test
4096        test
11024       test

# NO CHANGE IN SIZE.... HMM....

EDIT -

Let me further qualify that I don't want to rewrite files, copy files, etc. If it is not possible to somehow free previously allocated blocks in situ, so be it, but I'd like to determine if such is actually possible or not. It seems like "no, it is not" at this point. I suppose I'm looking for sys_punchhole for Linux (discussions of which I just stumbled upon).


Solution

  • Right now it appears that only NTFS supports hole-punching. This has been historically a problem across most filesystems. POSIX as far as I know, does not define an OS interface to punch holes, so none of the standard Linux filesystems have support for it. NetApp supports hole punching through Windows in its WAFL filesystem. There is a nice blog post about this here.

    For your problem, as others have indicated, the only solution is to move the file leaving out blocks containing zeroes. Yeah its going to be slow. Or write an extension for your filesystem on Linux that does this and submit a patch to the good folks in the Linux kernel team. ;)

    Edit: Looks like XFS supports hole-punching. Check this thread.

    Another really twisted option can be to use a filesystem debugger to go and punch holes in all indirect blocks which point to zeroed out blocks in your file (maybe you can script that). Then run fsck which will correct all associated block counts, collect all orphaned blocks (the zeroed out ones) and put them in the lost+found directory (you can delete them to reclaim space) and correct other properties in the filesystem. Scary, huh?


    Disclaimer: Do this at your own risk. I am not responsible for any data loss you incur. ;)