Read dictionary from file

Question

Background (optional)

I am writting a python script to analyse Abaqus (finite element software) outputs. This software generates a ".odb" which has a propriedtary format. However you can access the data stored inside of the databse thanks to python libraries specialy developped by Dassault (owner of the Abaqus sofware). The python script has to be run by the software in order to access these libraries :

abaqus python myScript.py

However it is really hard to use new libraries this way, and I cannot make it run the matplotlib library. So I would like to export the array I created inside a file, and access it later with an other script that would not require to be run using abaqus

The Problem

In order to manipulate the data, I am using collections. For exemple:

coord2s11=defaultdict(list)

This array stores the Z coordinate of a group of nodes and their stress value, at each time step:

coord2s11[time_step][node_number][0]=z_coordinate
coord2s11[time_step][node_number][1]=stress_value

For a given time step, the output would be :

defaultdict(<type 'list'>, {52101: [-61.83229635920749, 0.31428813934326172], 52102: [-51.948098314163417, 0.31094224750995636],[...], 52152: [440.18335942363655, -0.11255115270614624]})

And the glob (for all step time):

defaultdict(<type 'list'>, {0.0: defaultdict(<type 'list'>, {52101: [0.0, 0.0],[...]}), 12.660835266113281: defaultdict(<type 'list'>, {52101: [0.0, 0.0],[...],52152: [497.74876378582229, -0.24295337498188019]})})

If it is visually unpleasant, it is rather easy to use ! I printed this array inside this file using:

with open('node2coord.dat','w') as f :
    f.write(str(glob))

I tried to follow the solution I found on this post, but when I try to read the file a store the value inside a new dictionnay

import ast

with open('node2coord.dat', 'r') as f:
    s = f.read()
    node2coord = ast.literal_eval(s)

I end up with a SyntaxError: invalid syntax, that I guess comes from the defaultdict(<type 'list'> here and there in the array.

Is there a way to get the data stored inside of the file or should I modify the way it is written inside the file ? Ideally I would like to create the exact same array I stored.

The solution by Joel Johnson

Creating a database using shelve. It is an easy and fast method. The following code did the trick for me to create the db :

import os
import shelve

curdir = os.path.dirname(__file__) #defining current directory

d = shelve.open(os.path.join(curdir, 'nameOfTheDataBase')) #creation of the db
d['keyLabel'] = glob # storing the dictionary in "d" under the key 'keyLabel'
d.close() # close the db

The "with" statement did not work for me. And then to open it again :

import os
import shelve


curdir = os.path.dirname(__file__)
d = shelve.open(os.path.join(curdir, 'nameOfTheDataBase')) #opening the db
newDictionary = d['keyLabel'] #loading the dictionary inside of newDictionary
d.close()

If you ever get an error saying

ImportError: No module named gdbm

Just install the gdbm module. For linux :

sudo apt-get install python-gdbm

More information here

Answer 1

If you have access to shelve (which I think you would because it's part of the standard library) I would highly recommend using that. Using shelve is an easy way to store and load python objects without manually parsing and reconstructing them.

import shelve

with shelve.open('myData') as s:
    s["glob"] = glob

Thats it for storing the data. Then when you need to retrieve it...

import shelve

with shelve.open('myData') as s:
   glob = s["glob"]

It's as simple as that.