Pandas exporting and importing different dictionaries

Question

I have two functions for export and import dicts to and from csv, because I have many dictionaries of different kinds.

def export_dict(dict, filename):
    df = pd.DataFrame.from_dict(dict, orient="index")
    df.to_csv(CSV_PATH + filename)


def import_dict(filename):
    df = pd.read_csv(filename, header = None, keep_default_na=False)
    d = df.to_dict('split')
    d = dict(d["data"])
    return d

The dicts look like that:

d1 = {'123456': 8, '654321': 90, '123654': 157483}
d2 = {'ouwejfw': [4], 'uwlfhsn0': [3, 1, 89], 'afwefwe': [3, 4], 'a3rz0dsd': []}

So basically the first case is a simple one, where a string is a key (but it could be an int) and a number is the value. The second case is where a string is the key and a list of values of different sizes is the value. The first one, I can write and read without problems. But the second one breaks everything, because of the different sized lists. I cannot shorten them and I cannot use too much space by adding columns, because I have many Millions of data.

Can somebody help me, what can I do to read/write both dicts correctly? Thank you!

Answer 1

You can consider the dictionary values as a single item while storing in dataframe

import pandas as pd

def export_dict(dict, filename):
    df = pd.DataFrame()
    df["keys"] = d2.keys()
    df["values"] = d2.values()
    df.to_csv(CSV_PATH + filename)

d2 = {'ouwejfw': [4], 'uwlfhsn0': [3, 1, 89], 'afwefwe': [3, 4], 'a3rz0dsd': []}
export_dict(d2, "your_filename.csv")