Safest way to incrementally append to a file?

Question

I'm performing some calculations to generate chaotic solutions to a mathematical function. I have an infinite loop that looks something like this:

f = open('solutions.csv', 'a')
while True:
    x = generate_random_parameters() # x is a list of floats
    success = test_parameters(x)
    if success:
        print(','.join(map(str, x)), file=f, flush=True)

The implementation of generate_random_parameters() and test_parameters() is not very important here.

When I want to stop generating solutions I want to ^C, but I want to ensure that solutions.csv keeps its integrity/doesn't get corrupted/etc, in case I happen to interrupt when the file is being written to.

So far I haven't observed this happening, but I'd like to remove any possibility that this could occur.

Additionally, since the program will never terminate on its own I don't have a corresponding f.close() -- this should be fine, correct?

Appreciate any clarification.

Answer 1

One simple approach to ensuring that the current call to print finishes before the program exits from a keyboard interrupt is to use a signal handler to unset a flag on which the while loop runs.

Set the signal handler only when you're about to call print and reset the signal handler to the original when print returns, so that the preceding code in the loop can be interrupted normally:

import signal

def interrupt_handler(signum, frame):
    global running
    running = False

text = 'a' * 99999
running = True
with open('solutions.csv', 'a') as f:
    while running:
        ... # your calculations
        original_handler = signal.signal(signal.SIGINT, interrupt_handler)
        print(text, file=f, flush=True) # your output
        signal.signal(signal.SIGINT, original_handler)

Also note that it is more idiomatic to use open as a context manager to handle the closure of an open file when exiting the block for any reason.