Dumping data with pickle

Question

I am very confused about how to use pickle, I'm trying to make the computation a lot faster by storing the values that have been computed, and creating a pickle file for the ones computed. When a certain function hasn't been computed yet I want to open the pickle file and create a new input of the newly computed function. Here's my atttempt:

try:
    with open('EinsteinBbb.pickle','rb') as f:
        Bbb = pickle.load(f)
except:
    Bbb = dict()
def EinsteinBbb(nl, ll, nu, lu):

    global Bbb
    try:
        Bbbfile=pickle.load(open('EinsteinBbb.pickle','rb'))
    # Check if Bbb[(nl, ll, nu, lu)] needs to be computed
        if True: #TODO: replace with the correct condition
            print('Computing Bbb[{0}, {1}, {2}, {3}]'.format(nl, ll, nu, lu))
    except:   
        Bbb[(nl, ll, nu, lu)] = nl*ll*nu*lu
        with open('EinsteinBbb.pickle','wb') as f:
            pickle.dump(Bbb, f)
        with open('EinsteinBbb.pickle','rb') as f:
            EinsteinBbb_read = pickle.load(f)
        # TODO: Open 'EinsteinBbb.pickle' with write mode and dump Bbb so that it does not need to be recomputed
    return Bbb[(nl, ll, nu, lu)]
print(EinsteinBbb(1,2,3,4))

When I try for example print(EinsteinBbb(2,1,2,0)) I get TypeError: string indices must be integers I feel like my code is very far from being right but any advice would be incredibly helpful!

Edit: Creating a minimal reproducible made the TypeError disappear, and I think it had to do with where the code was stored, but now I get KeyError for any key other than the one I ran the code with the first time.

Answer 1

I think the following does what you want (including all your TODOs as well as not producing a KeyError for keys not already in the Bbb dictionary). It uses exceptions to determine when it can initialize the dictionary from the existing pickled cache file, as well as when an attempt is being made to access a non-existent entry so an new entry for it gets added.

I put in number of print calls to enable you to see what it's doing.

Tip: Most of the time you should avoid using "bare" except: statements when handling exceptions because they can hide things you would probably like to allow to occur, like SyntaxErrors or SIGINT keyboard (Ctrl + C) signals, and therefore would not want suppressed. In situations where you're not exactly sure what to expect, use except Exception: instead to avoid the problem of catching too much (because it won't include them).

import pickle


PICKLE_FILEPATH = 'EinsteinBbb.pickle'

# Load or create cache dictionary
try:
    with open(PICKLE_FILEPATH, 'rb') as f:
        Bbb = pickle.load(f)
    print('Bbb dictionary read from file.')
except Exception:
    print('Creating Bbb dictionary.')
    Bbb = dict()

def EinsteinBbb(nl, ll, nu, lu):
    try:
        return_value = Bbb[(nl, ll, nu, lu)]
        print(f'Using existing entry for ({nl}, {ll}, {nu}, {lu}) in Bbb.')
    except KeyError:
        print(f'Adding entry ({nl}, {ll}, {nu}, {lu}) to Bbb.')
        return_value = Bbb[(nl, ll, nu, lu)] = nl*ll*nu*lu
        # Overwrite pickle file with updated dictionary.
        with open(PICKLE_FILEPATH, 'wb') as f:
            pickle.dump(Bbb, f)

    return return_value

print(f'{EinsteinBbb(1,2,3,4)=}')
#print(f'{EinsteinBbb(2,3,4,5)=}')