Search code examples
pythonnumpymmaptemporary-files

Numpy memmap with file deletion


Is it possible to have a Numpy memmap file who's file will be deleted when the memmap array is garbage collected?

I have tried:

import tempfile
import numpy as np
arr = np.memmap(tempfile.NamedTemporaryFile(), mode='w+',
                shape=(10, 10), dtype=np.int)
os.path.exists(arr.filename) # False

But it appears the reference to the temporary file isn't kept so is deleted.

I don't want to use a context manager on the temporary file, as I want to be able to return the array from a function and have the file live until the array is no longer used.

NB: similar question here: In Python, is it possible to overload Numpy's memmap to delete itself when the memmap object is no longer referenced? but the asker exhibits poor knowledge of Python scoping and the tempfile module.


Solution

  • As it turns out, jtaylor's answer to the question originally linked is correct. So the code:

    import tempfile
    import numpy as np
    with tempfile.NamedTemporaryFile() as ntf:
        temp_name = ntf.name
        arr = np.memmap(ntf, mode='w+',
                        shape=(10, 10), dtype=np.int)
    print(arr)
    

    Works as desired, even though os.path.exists(temp_name) is false, because of the way the OS manages files. The file path (temp_name) is unlinked, and no longer accessible through the filesystem, however the actual file disk usage will remain available until the open file is closed, which the memmap object will keep.