large getitem overhead when accessing internal dictionary?

I have the following map class that uses a dictionary as it's base data structure:

class Map:

''' A map object to handle the game world
'''

    def __init__(self, initial_grid=None, rooms=None, default=None):

        self.grid = initial_grid
        if not self.grid:
            self.grid = {}

        self.default = default

And it has an access method that tries to get the info from it's grid, but if it can't returns the default argument

    def __getitem__(self, key):
        return self.grid.get(key, self.default)

Now in another part of my code, I have this function running a very large number of time, giving me the following cProfile results:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)

2510000    0.739    0.000    0.926    0.000 map.py:57(__getitem_ _)

2510001    0.187    0.000    0.187    0.000 {method 'get' of 'dict' objects}

If I understand it correctly, most of the time in the function (~80%) is not spent on retrieving the value, so what actually happens in all that time? Any way of improving that?

Solution

There is a lot of dict getting involved: self.grid is a __getitem(…), which gets the grid from the self-dict, then self.grid.get is another one, finally self.default gets the default entry from self.

To reduce the number of gets, you may want to add the following to your class:

__slots__ = ('grid', 'default') # add whatever field you have

This should speed up your code considerably, because slot access is reduced to a simple array access.

large __getitem__ overhead when accessing internal dictionary?

large getitem overhead when accessing internal dictionary?