Python 3: Flatten Dictionary including lists

Question

Is there an elegant way to achieve this in python 3? It is supposed to flatten the dictionary, while treating list indices like keys and thus including them in the flattening.

in:

{
  "a": 1,
  "b": {
    "c": 2
  },
  "d": [3,4],
  "e": [
    {
      "f": 5
    },
    {
      "g": 6
    }
  ]
}

out:

{
  "a": 1,
  "b.c": 2,
  "d.0": 3,
  "d.1": 4,
  "e.0.f": 5,
  "e.1.g": 6
}

Background:

• We are looking for a way to merge nested dictionaries containing lists
• The available merge tools seem to always merge lists with either an append strategy or with a replace strategy
• We however need to merge dicts that are inside lists (as in the example). E.g., the first dict in the list should be merged with the first dict in the list
• The current approach is to chain like this: flatten -> merge -> unflatten. The question was only about the flatten piece. We have since seen that the unflatten is trickier
• Optimally, we use already existing robust libraries

Answer 1

A possibiliy would be to make use of the flatten-json library.

It provides a flatten, unflatten and unflatten_list function.

In your usecase the flatten und unflatten_list functions are needed.

from flatten_json import flatten, unflatten_list

d = {
  "a": 1,
  "b": {
    "c": 2
  },
  "d": [3,4],
  "e": [
    {
      "f": 5
    },
    {
      "g": 6
    }
  ]
}

desired_result = {
  "a": 1,
  "b.c": 2,
  "d.0": 3,
  "d.1": 4,
  "e.0.f": 5,
  "e.1.g": 6
}


d_flat = flatten(d, separator=".")
print(d_flat == desired_result)  # True

d_unflat = unflatten_list(d_flat, separator=".")
print(d == d_unflat)  # True