I'm working on a class representing on object with numerous associated data. I'm storing these data in a dictionary class attribute called metadata. A representation could be:
{'key1':slowToComputeValue, 'key2':evenSlowerToComputeValue}
The calculating of the values is in some cases very slow, so what I want to do is, using "getter" functions, first try and get the value from the metadata dict. Only on a KeyError (i.e. when the getter tries to get a value for a key which doesn't exist yet) should the value be calculated (and added to the dictionary for fast access next time the getter is called).
I began with a simple:
try:
return self.metadata[requested_key]
except KeyError:
#Implementation of function
As there are many getters in the class, I started thought that these first 3 lines of code could be handled by a decorator. However I'm having problems making this work. The problem is that I need to pass the metadata dictionary from the class instance to the decorator. I've found several tutorials and posts like this one which show that it is possible to send a parameter to an enclosing function but the difficulty I'm having is sending a class instantiation attribute metadata to it (if I send a string value it works).
Some example code from my attempt is here:
def get_existing_value_from_metadata_if_exists(metadata):
def decorator(function):
@wraps(function)
def decorated(*args, **kwargs):
function_name = function.__name__
if function_name in metadata.keys():
return metadata[function_name]
else:
function(*args, **kwargs)
return decorated
return decorator
class my_class():
@get_existing_value_from_metadata_if_exists(metadata)
def get_key1(self):
#Costly value calculation and add to metadata
@get_existing_value_from_metadata_if_exists(metadata)
def get_key2(self):
#Costly value calculation and add to metadata
def __init__(self):
self.metadata = {}
The errors I'm getting are generally self not defined but I've tried various combinations of parameter placement, decorator placement etc. without success.
So my questions are:
Yes, a decorator is a good use case for this. Django for example has something similar already included with it, it's called cached_property.
Basically all it does is that when the property is accessed first time it will store the data in instance's dict(__dict__) by the same name as the function. When we fetch the same property later on it simple fetches the value from the instance dictionary.
A cached_property is a non-data descriptor. Hence once the key is set in instance's dictionary, the access to property would always get the value from there.
class cached_property(object):
"""
Decorator that converts a method with a single self argument into a
property cached on the instance.
Optional ``name`` argument allows you to make cached properties of other
methods. (e.g. url = cached_property(get_absolute_url, name='url') )
"""
def __init__(self, func, name=None):
self.func = func
self.__doc__ = getattr(func, '__doc__')
self.name = name or func.__name__
def __get__(self, instance, cls=None):
if instance is None:
return self
res = instance.__dict__[self.name] = self.func(instance)
return res
In your case:
class MyClass:
@cached_property
def key1(self):
#Costly value calculation and add to metadata
@cached_property
def key2(self):
#Costly value calculation and add to metadata
def __init__(self):
# self.metadata not required
Use the name argument to convert an existing method to cached property.
class MyClass:
def __init__(self, data):
self.data = data
def get_total(self):
print('Processing...')
return sum(self.data)
total = cached_property(get_total, 'total')
Demo:
>>> m = MyClass(list(range(10**5)))
>>> m.get_total()
Processing...
4999950000
>>> m.total
Processing...
4999950000
>>> m.total
4999950000
>>> m.data.append(1000)
>>> m.total # This is now invalid
4999950000
>>> m.get_total() # This still works
Processing...
4999951000
>>> m.total
4999950000
Based on the example above we can see that we can use total as long as we know the internal data hasn't been updated yet, hence saving processing time. But it doesn't make get_total() redundant, as it can get the correct total based on the data.
Another example could be that our public facing client was using something(say get_full_name()) as method so far but we realised that it would be more appropriate to use it as a property(just full_name), in that case it makes sense to keep the method intact but mark it as deprecated and start suggesting the users to use the new property from now on.