I am trying to read in a very large (several GB) binary file with numpy.fromfile(). Reading in the entire file at once generates an out of memory error, so I want to create a loop to read and process N chunks of data at a time. Something like the following:
while True:
numpy.fromfile(f, recordType, N)
# proccess data
if f.EOF():
break
How do I detect when I have reached the end of the file, so that I can break my loop?
while True:
a = numpy.fromfile(f, recordType, N)
# proccess data
if a.size < N:
break