A particular problem with unit tests. Many of my test function have the following structure:
def test_xxx():
try:
# do-something
variable1 = ...
variable2 = ...
except Exception as error:
raise error
finally:
try:
del variable1
except Exception:
pass
try:
del variable2
except Exception:
pass
And that structure is obviously not very nice. I could simplify the finally statement with:
finally:
for variable in ("variable1", "variable2"):
if variable in locals():
del locals()[variable]
but it's still not that great. Instead, I would like to use a decorator del_variables to handle the try / except / finally.
Attempt, not working:
def del_variables(*variables):
def decorator(function):
def wrapper(*args, **kwargs):
try:
return function(*args, **kwargs)
except Exception as error:
raise error
finally:
for variable in variables:
if variable in locals():
del locals()[variable]
return wrapper
return decorator
@del_variables("c")
def foo(a, b):
c = 3
return a + b + c
Since locals() doesn't refer to the namespace inside the execution of function. If I use vars(), I don't know what argument I should provide since that namespace as far as Python is concerned is already GC. Any idea how I could get that decorator to work, i.e. how I could explicitly delete variables from the namespace of function?
Notes: yes, this is a weird case where I need to explicitly call del because I overwrote the __del__ methods of certain objects (stored in the variables I want to delete) to call c++ functions to destroy the c++ objects and references attached. And no, pytest fixtures are not a good solution in that case ;)
You should really consider restructuring your code. Relying on finalizers is not a good idea to begin with (see e.g. this Q&A). There are better mechanisms available for resource cleanup like context managers and/or pytest fixtures.
That being said:
If a test fails, pytest retains the raised exception. That exception contains all stack frames. The stack frame of the test function contains references to all objects created in the test.
With your current solution, you only delete the references in the test function. However, there may be more references to the objects in other stack frames. For example:
def func(some_obj):
raise Exception
def test_test():
obj = MyObj()
func(obj)
del obj won't cause the object to be garbage collected, because now func's stack frame still contains a reference to the object.
Since you cannot reasonably trace the references to your object through the call stack (it may be contained in dicts inside other objects etc.), I see no simple way to only delete selected objects like you requested. However, assuming that there are no global or cyclic references to the objects, you can simply delete all locals from the subsequent frames. Oh, and we're only talking about CPython, of course.
This can be achieved in multiple ways, but since you wanted a decorator:
import ctypes
import sys
def delete_locals(func):
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
finally:
tb = sys.exc_info()[2]
if tb:
while tb.tb_next:
tb = tb.tb_next
tb.tb_frame.f_locals.clear()
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(tb.tb_frame), ctypes.c_int(1)
) # reference: https://stackoverflow.com/a/34671307
return wrapper
Keep in mind that pytest now also has no chance to tell you the values of locals in its output.