I made up this simple, contrived example of some code I ran into at work. I'm trying to better understand why slow_function_1 (+ the way its decorators are structured) would cache function results properly, but the decorator applied to slow_function_2 would not. In this example, I'm trying to access cache information after calling the method; however, I consistently get the following error: AttributeError: 'function' object has no attribute 'cache_info'
. I've searched high and low to try to fix this, but to no avail. This AttributeError is raised for both slow_function_1.cache_info()
and slow_function_2.cache_info()
How do I view the cache between function calls? If anyone has any insight on the original problem of why slow_function_1 and slow_function_2 differ in caching behavior, I would appreciate that as well.
Thank you in advance!
import functools
import time
def format_args(func):
def inner(*args, **kwargs):
formatted_args = [tuple(x) if type(x) == list else x for x in args]
return func(*formatted_args, **kwargs)
return inner
def formatted_cache(func):
def inner(*args, **kwargs):
formatted_args = [tuple(x) if type(x) == list else x for x in args]
return functools.lru_cache()(func)(*formatted_args, **kwargs)
return inner
@format_args
@functools.lru_cache
def slow_function_1(a: list, b: bool):
time.sleep(1)
print("executing slow function 1")
return sum(a)
@formatted_cache
def slow_function_2(a: list, b: bool):
time.sleep(1)
print("executing slow function 2")
return functools.reduce((lambda x, y: x*y), a)
example_list = [1,2,3,4,5,6,7,8,9,10,11,12]
example_bool = True
slow_function_1(example_list, example_bool)
print(slow_function_1.cache_info())
slow_function_1(example_list, example_bool)
print(slow_function_1.cache_info())
slow_function_2(example_list, example_bool)
print(slow_function_2.cache_info())
slow_function_2(example_list, example_bool)
print(slow_function_2.cache_info())
Now that I stared at it for a good time, I don't think it's really possible to do this with a decorator. You need a lru_cache
object to access the cache and all that stuff, and you need a second function to format the arguments to be hashable before passing to the lru_cache
object. The decorator can't return both at once, and they can't be nested in each other to make one function with the best of both worlds.
def formatted_cache(func):
# first we assume func only takes in hashable arguments
# so cachedfunc only takes in hashable arguments
cachedfunc = functools.lru_cache(func)
# inner formats lists to hashable tuples
# then passes it to cachedfunc
def inner(*args, **kwargs):
formatted_args = [tuple(x) if type(x) == list else x for x in args]
return cachedfunc(*formatted_args, **kwargs)
# oh no, we can only return one function, but neither is good enough
I think the only way to move forward is to just accept that these have to be done in separate functions because of lru_cache
's limitation. It's not that awkward, actually, just a simple higher order function like map
.
import functools
import time
def formatted_call(func, *args, **kwargs):
formatted_args = [tuple(x) if type(x) == list else x for x in args]
return func(*formatted_args, **kwargs)
@functools.lru_cache
def slow_function_2(a: list, b: bool):
time.sleep(1)
print("executing slow function 2")
return functools.reduce((lambda x, y: x*y), a)
example_list = [1,2,3,4,5,6,7,8,9,10,11,12]
example_bool = True
formatted_call(slow_function_2, example_list, example_bool)
print(slow_function_2.cache_info())
formatted_call(slow_function_2, example_list, example_bool)
print(slow_function_2.cache_info())