Dealing with EOFError: Ran out of input error from file that is constantly updated

Question

I have the following code that loads a pickle file frequently throughout the day:

import pandas as pd

df = pd.read_pickle('data')

Sometimes I get the following error:

EOFError: Ran out of input

I assume that this happens when the file is in the middle of being updated by a different program. To fix, this I tried the following:

import pandas as pd
import time

try:
    df = pd.read_pickle('data')
except:
    time.sleep(0.5)
    df = pd.read_pickle('data)

I get the error less frequently, but eventually I get it again as the "data" file is constantly updated, more than once per second.

Is there a way of making a loop that keeps trying for say 10 or 20 trys before throwing an error? Without just writing 10 or 20 nested trys and excepts?

Answer 1

To answer your question about 20 tries, you can use a loop. Make sure to catch the specific exception, in case a different error is thrown.

for i in range(20):
    try:
        df = pd.read_pickle('data')
    except EOFError:
        time.sleep(0.5)