Search code examples
pythonpython-3.xnumpyio

loading big .npy file causing python to stop working

I am trying to load a .npy file which is quite big (6GB) and I get a Python has stopped working error. Any suggestions as to how to approach getting around this issue? enter image description here

Solution

  • You can use buffering for bigger files eg:

    with open("bigfile.npy","r",buffering=1000) as f:
    contents=f.read()
    #do something with the fist n lines of file at a time

    this willn't open the entire file altogether but instead only the quantity that you specify as integer in buffering arg (you can decide on what you want depending on the system you are working on) and the system will not stop working.