Search code examples
pythonnumpywith-statement

Should np.load() be in with with open


Traditionally in python one user with to open files such that the file automatically closes when you exit the inner scope.

with open('file.pickle', 'rb') as f:
    data = pickle.load(f)

In the numpy documentation it shows both np.load and with load

with load('foo.npz') as data:
    a = data['a']

a = np.load('/tmp/123.npy')

It looks like the with method only works with .npz files. Does a=np.load() automatically close the data file just like when the with context is closed?


Solution

  • Using random files from my current work dir:

    Using with with a npz:

    In [51]: with np.load('f.npz') as f:print(f['b'])
    ['q' 'r' 's']
    

    Using a 'plain' load with a npy:

    In [55]: a = np.load('adict.npy',allow_pickle=True)
    In [56]: a
    Out[56]: 
    array({'#Ineligiblevets': array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), 'test': 'one'},
          dtype=object)
    

    a is an array, so can't be "closed":

    In [57]: a.close()
    Traceback (most recent call last):
      File "<ipython-input-57-3980b52f6377>", line 1, in <module>
        a.close()
    AttributeError: 'numpy.ndarray' object has no attribute 'close'
    

    It is possible to open the file before hand, and pass via with, but you have to get the modes right, so it really doesn't help (unless you are doing something special):

    In [59]: with open('adict.npy', 'rb') as f:
        ...:     a = np.load(f, allow_pickle=True)
        ...: 
    In [60]: a
    Out[60]: 
    array({'#Ineligiblevets': array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), 'test': 'one'},
          dtype=object)
    

    Trying to use a npy load in a with doesn't work because an array does not have required enter/exit methods:

    In [61]: with np.load('adict.npy', allow_pickle=True) as data: a=data
    Traceback (most recent call last):
      File "<ipython-input-61-1025c4c8c4a8>", line 1, in <module>
        with np.load('adict.npy', allow_pickle=True) as data: a=data
    AttributeError: __enter__