Search code examples
pythonnumpyfromfile

How to detect EOF with numpy.fromfile


I am trying to read in a very large (several GB) binary file with numpy.fromfile(). Reading in the entire file at once generates an out of memory error, so I want to create a loop to read and process N chunks of data at a time. Something like the following:

while True:
   numpy.fromfile(f, recordType, N)
   # proccess data 
   if f.EOF():
        break

How do I detect when I have reached the end of the file, so that I can break my loop?


Solution

  • while True:
       a = numpy.fromfile(f, recordType, N)
       # proccess data 
       if a.size < N:
            break