I need to append to the beginning or middle of a binary file and that file may be large (anywhere from 100Mb to 15Gb). Most of the answers I've seen so far either do not accommodate large files or basically rewrite the file entirely. Hoping to avoid "writing to a new file and replacing old file" method if at all possible. In my head, something like this should be able to add data from, for example, the 60th byte:
with open('file.dat', 'ab') as f:
f.seek(60)
f.write(b'_hello_world_')
However it doesn't work as intended, rather it appends to the end of the file. Still trying to wrap my head around how the previous example fails, but using io.BytesIO() works in the same fashion. Hopefully I'm just overlooking something simple.
The filemode parameter a means explicitely to append text at the end of the line, see documentation:
and 'a' for appending (which on some Unix systems, means that all writes append to the end of the file regardless of the current seek position)
So seeking does not help if you use a as a filemode parameter. But you don't have to use a. Simply use r+ instead, that means to open file with the possibility to update it also (read and write):
with open('file.dat', 'r+b') as f:
f.seek(60)
f.write(b"_hello_world_")
I'm still not sure if this is the case for Windows, too (documentation said "on some Unix systems").