Reading a binary file in Python into a struct

Question

How do I go about opening a binary data file in Python and reading back the values one long at a time, into a struct. I have something like this at the moment but I think this will keep overwriting idList, I want to append to it, so I end up with a tuple of all the long values in the file -

file = open(filename, "rb")
    try:
        bytes_read = file.read(struct.calcsize("=l"))
        while bytes_read:
            # Read 4 bytes(long integer)
            idList = struct.unpack("=l", bytes_read)
            bytes_read = file.read(struct.calcsize("=l"))
    finally:
        file.close()

Answer 1

Simplest (python 2.6 or better):

import array
idlist = array.array('l')
with open(filename, "rb") as f:
    while True:
        try: idlist.fromfile(f, 2000)
        except EOFError: break
idtuple = tuple(idlist)

Tuples are immutable, so they can't be built incrementally: so you have to build a different (mutable) sequence, then call tuple on it at the end. If you don't actually need specifically a tuple, of course, you can save the last, costly step and keep the array or list or whatever. Avoiding trampling over built-in names like file is advisable anyway;-).

If you have to use the struct module for a job that's best handled by the array module (e.g., because of a bet),

idlist = [ ]
with open(filename, "rb") as f:
    while True:
        bytes_read = f.read(struct.calcsize("=l"))
        if not bytes_read: break
        oneid = struct.unpack("=l", bytes_read)[0]
        idlist.append(oneid)

The with statement (also available in 2.5 with an import form the future) is better than the old try/finally in clarity and conciseness.