I seem to be confused about how file removal works on Linux (same behavior on Mac OS X). The following code writes a 1GB file and then immediately deletes it. It does so 1024 times. I thought that since the file is closed, it would be immediately removed (space reclaimed and maybe even the file descriptor recycled). Instead, if I look at lsof, the file is still claimed as owned by the process (with (deleted) appended to it). If I look at df, the space is still used up. Eventually, the process dies from either exhausting the number of allowed open files, a limit I can raise, or just running out of space.
What's the right way to do this? Can I actually delete the file before the process terminates (and reclaim its space), or is this a hopeless exercise and I need another design for my code?
#include <unistd.h>
#include <cstdlib>
#include <cstdio>
#include <fcntl.h>
#include <iostream>
int main()
{
size_t n = 1024*1024*1024;
void* buffer = malloc(n);
std::cout << "Buffer created" << std::endl;
for (unsigned i = 0; i < 1024; ++i)
{
char filename[50] = "STORAGE.XXXXXX";
mkstemp(filename);
std::cout << filename << std::endl;
int fh = open(filename, O_WRONLY | O_SYNC, 0600);
write(fh, buffer, n);
close(fh);
remove(filename);
}
free(buffer);
sleep(60);
}
lsof:
test-remo 29251 ... 3u REG 8,51 1073741824 9307914 .../tmp/STORAGE.Tyj4oG (deleted)
test-remo 29251 ... 4u REG 8,51 1073741824 9308254 .../tmp/STORAGE.fBkF5b (deleted)
test-remo 29251 ... 5u REG 8,51 1073741824 9308030 .../tmp/STORAGE.xfJ9P0 (deleted)
test-remo 29251 ... 6u REG 8,51 1073741824 9308538 .../tmp/STORAGE.0C6oea (deleted)
test-remo 29251 ... 7u REG 8,51 1073741824 9306147 .../tmp/STORAGE.w9dPzC (deleted)
test-remo 29251 ... 8u REG 8,51 1073741824 9308311 .../tmp/STORAGE.mIwcpq (deleted)
test-remo 29251 ... 9u REG 8,51 1073741824 9309004 .../tmp/STORAGE.srTZfy (deleted)
test-remo 29251 ... 10u REG 8,51 1073741824 9309284 .../tmp/STORAGE.xjQQ4Y (deleted)
test-remo 29251 ... 11u REG 8,51 1073741824 9309291 .../tmp/STORAGE.ioyPxI (deleted)
test-remo 29251 ... 12u REG 8,51 1073741824 9307415 .../tmp/STORAGE.saYo2L (deleted)
test-remo 29251 ... 13u REG 8,51 1073741824 9307810 .../tmp/STORAGE.ylW467 (deleted)
test-remo 29251 ... 14u REG 8,51 1073741824 9308666 .../tmp/STORAGE.lUYnWN (deleted)
test-remo 29251 ... 15u REG 8,51 1073741824 9309678 .../tmp/STORAGE.1rZM4M (deleted)
test-remo 29251 ... 16u REG 8,51 1073741824 9309702 .../tmp/STORAGE.wdggra (deleted)
test-remo 29251 ... 17u REG 8,51 1073741824 9309687 .../tmp/STORAGE.qczYAS (deleted)
test-remo 29251 ... 18u REG 8,51 1073741824 9309732 .../tmp/STORAGE.u0IxKV (deleted)
test-remo 29251 ... 19u REG 8,51 1073741824 9309708 .../tmp/STORAGE.6SiMpi (deleted)
test-remo 29251 ... 20u REG 8,51 1073741824 9308649 .../tmp/STORAGE.URky5Z (deleted)
test-remo 29251 ... 21u REG 8,51 1073741824 9309740 .../tmp/STORAGE.j8wEq2 (deleted)
test-remo 29251 ... 22u REG 8,51 1073741824 9309677 .../tmp/STORAGE.Wj5rco (deleted)
test-remo 29251 ... 23u REG 8,51 1073741824 9309828 .../tmp/STORAGE.I38ln3 (deleted)
test-remo 29251 ... 24u REG 8,51 1073741824 9309839 .../tmp/STORAGE.ent8I1 (deleted)
test-remo 29251 ... 25u REG 8,51 1073741824 9309843 .../tmp/STORAGE.1361nk (deleted)
test-remo 29251 ... 26u REG 8,51 1073741824 9309824 .../tmp/STORAGE.ue2WkW (deleted)
test-remo 29251 ... 27u REG 8,51 1073741824 9312037 .../tmp/STORAGE.4MdKsR (deleted)
test-remo 29251 ... 28u REG 8,51 1073741824 9312042 .../tmp/STORAGE.EgDft5 (deleted)
test-remo 29251 ... 29u REG 8,51 1073741824 9312059 .../tmp/STORAGE.NRcqlE (deleted)
test-remo 29251 ... 30u REG 8,51 1073741824 9309847 .../tmp/STORAGE.cF9qYw (deleted)
test-remo 29251 ... 31u REG 8,51 1073741824 9312103 .../tmp/STORAGE.wzfhSI (deleted)
test-remo 29251 ... 32u REG 8,51 1073741824 9312120 .../tmp/STORAGE.hfdv1d (deleted)
test-remo 29251 ... 33u REG 8,51 1073741824 9312090 .../tmp/STORAGE.kSX2D4 (deleted)
test-remo 29251 ... 34u REG 8,51 1073741824 9309813 .../tmp/STORAGE.FWKVGf (deleted)
test-remo 29251 ... 35u REG 8,51 1073741824 9312148 .../tmp/STORAGE.CYPGGL (deleted)
test-remo 29251 ... 36u REG 8,51 1073741824 9312171 .../tmp/STORAGE.it4bmC (deleted)
test-remo 29251 ... 37u REG 8,51 1073741824 9312145 .../tmp/STORAGE.IGSzVO (deleted)
test-remo 29251 ... 38u REG 8,51 1073741824 9312228 .../tmp/STORAGE.CMSOil
When you're invoking mkstemp, it not only creates a temporary file, it also opens it and returns a filehandle.
#include <unistd.h>
#include <cstdlib>
#include <cstdio>
#include <fcntl.h>
#include <iostream>
int main()
{
size_t n = 1024*1024*1024;
void* buffer = malloc(n);
std::cout << "Buffer created" << std::endl;
for (unsigned i = 0; i < 1024; ++i)
{
char filename[50] = "STORAGE.XXXXXX";
int fh;
fh = mkstemp(filename);
// close(fh)
std::cout << filename << std::endl;
// fh = open(filename, O_WRONLY | O_SYNC, 0600);
write(fh, buffer, n);
close(fh);
remove(filename);
}
free(buffer);
sleep(60);
}
If you want to open it with WRONLY and SYNC, then you have to first close the returned filehandle, and then re-open it, if you so desire. In that case, just remove the // comment marks before close and open, or use mkostemp instead.
NAME
mkstemp, mkostemp, mkstemps, mkostemps - create a unique temporary file
SYNOPSIS
#include <stdlib.h>
int mkstemp(char *template);
int mkostemp(char *template, int flags);
RETURN VALUE
On success, these functions return the file descriptor of the temporary file.
On error, -1 is returned, and errno is set appropriately.