Search code examples
pythondictionarypandassparse-matrix

Convert sparse dictionary representation into a dense dataframe

I would like to convert a list of dictionaries which sparsely represent individual observations of features into a dense data structure (e.g. a dataframe).

Each observation is a dictionary with variable number of key(signal_name)-value depending on which signal has fired at a particularly instance.

I would like to convert this list of dictionaries into a dense dataframe such that the columns contain all possible keys. I began writing some code, but thought that I'd ask first if this functionality actually exists in a package somewhere.

Thanks.

Solution

  • I'm not sure what you've tried but this seems to Just WorkTM.

    In [11]: d1 = {'A': 1}

In [12]: d2 = {'A': 4, 'B': 5}

In [13]: d3 = {'C': 9}

In [14]: pd.DataFrame([d1, d2, d3])
Out[14]:
    A   B   C
0   1 NaN NaN
1   4   5 NaN
2 NaN NaN   9