Search code examples
pythonsaving-data

How to save multiple data at once in Python


I am running a script which takes, say, an hour to generate the data I want. I want to be able to save all of the relevant variables to some external file so I can fiddle with them later without having to run the hour-long calculation over again. Is there an easy way I can save all of the variables I need into one convenient file?

In Matlab I would just contain all of the results of the calculation in a single structure so that later I could just load results.mat and I would have everything I need stored as results.output1, results.output2 or whatever. What is the Python equivalent of this?

In particular, the data that I would like to save includes arrays of complex numbers, which seems to present difficulties for using things like json.


Solution

  • I suggest taking look at built-in shelve module which provides persistent, dictionary-like object and generally does work with all native Python types so you can do:

    Write complex to some file (in my example it is named mydata) under key n (keep in mind that keys should be strings).

    import shelve
    my_number = 2+7j
    with shelve.open('mydata') as db:
        db['n'] = my_number
    

    Later retrieve that number from given file

    import shelve
    with shelve.open('mydata') as db:
        my_number = db['n']
    print(my_number)  # (2+7j)