Search code examples
python-3.xdictionarydefaultdict

Erroneous behaviour while updating nested dictionary python3


While working on defaultdict class of collection package in python3.7, I see that new key is generated from the duplicate of last key, instead of initiating dictionary. Is there a way to initiate new element with given dictionary which is init_dict in below example code.

Example code to reproduce error:

from collections import defaultdict
init_dict = {'buy_qty': 0, 
             'sell_qty': 0}

pnl = defaultdict(lambda: init_dict)
pnl['a']['buy_qty'] += 1
pnl['a']['sell_qty'] += 1

Now when I do

pnl['b'] 

gives me

{'buy_qty': 1, 'sell_qty': 1}

I am looking for pnl['b'] to be initialized with init_dict. How can I achieve that?


Solution

  • Your copying by reference, not by value. So whatever you do to one dictionary, the other will be affected.

    You can check this with the id() function:

    print(id(pnl['a']))
    print(id(pnl['b']))
    
    print(id(pnl['a']) == id(pnl['b']))
    

    Which will give the same memory addresses:

    1817103232768
    1817103232768
    True
    

    verifying that they are the same objects. You can fix this by assigning a shallow copy of the dictionary using dict.copy(), as mentioned in the comments:

    pnl = defaultdict(lambda: init_dict.copy())
    

    Or casting dict():

    pnl = defaultdict(lambda: dict(init_dict))
    

    Or using ** from PEP 448 -- Additional Unpacking Generalizations :

    pnl = defaultdict(lambda: {**init_dict})
    

    Additionally, consider using a collections.Counter to do the counting, instead of initializing zero count dictionaries yourself:

    from collections import defaultdict, Counter
    
    pnl = defaultdict(Counter)
    
    pnl['a']['buy_qty'] += 1
    pnl['a']['sell_qty'] += 1
    
    print(pnl)
    # defaultdict(<class 'collections.Counter'>, {'a': Counter({'buy_qty': 1, 'sell_qty': 1})})
    
    print(pnl['b']['buy_qty'])
    # 0
    
    print(pnl['b']['buy_qty'])
    # 0
    
    pnl['b']['buy_qty'] += 1
    pnl['b']['sell_qty'] += 1
    
    print(pnl)
    # defaultdict(<class 'collections.Counter'>, {'a': Counter({'buy_qty': 1, 'sell_qty': 1}), 'b': Counter({'buy_qty': 1, 'sell_qty': 1})})
    

    Counter is a subclass of dict, so they will work the same as normal dictionaries.