How to test for a reference cycle caused by saved exception?

Question

I'm talking about this problem: https://bugs.python.org/issue36820.

Small summary:

Saving an exception causes a cyclic reference, because the exception's data include a traceback containing the stack frame with the variable where the exception was saved.

try:
    1/0
except Exception as e:
    ee = e

The code is not broken, beacuse Python will eventually free the memory with its garbage collector. But the whole sitation can be avoided:

try:
    1/0
except Exception as e:
    ee = e
    ...
    ...
finally:
    ee = None

---

In the linked bpo-36820 there is a demonstration with a weak reference kept alive.

My question is if there exist a test that does not need to edit the function itself. Something like

• run the tested function
• check if a new cycle was created

Can the gc module do that?

Answer 1

Yes, using the gc module, we can check whether there are (new) exceptions that are only referred to by a traceback frame.

In practice, iterating gc objects creates an additional referrer (can't use WeakSet as built-in exceptions don't support weakref), so we check that there are two referrers — the frame and the additional referrer.

def get_exception_ids_with_reference_cycle(exclude_ids=None):
    import gc
    import types
    exclude_ids = () if exclude_ids is None else exclude_ids
    exceptions = [
        o for o in gc.get_objects(generation=0)
        if isinstance(o, Exception) and id(o) not in exclude_ids
    ]
    exception_ids = [
        id(e) for e in exceptions
        if len(gc.get_referrers(e)) == 2 and all(
            isinstance(r, types.FrameType) or r is exceptions
            for r in gc.get_referrers(e)
        )
    ]
    return exception_ids

Usage:

exception_ids = get_exception_ids_with_reference_cycle()
x()
print(bool(get_exception_ids_with_reference_cycle(exclude_ids=exception_ids)))

Alternative usage:

@contextlib.contextmanager
def make_helper():
    exception_ids = get_exception_ids_with_reference_cycle()
    yield lambda: bool(get_exception_ids_with_reference_cycle(exclude_ids=exception_ids))


with make_helper() as get_true_if_reference_cycle_was_created:
    x()
    print(get_true_if_reference_cycle_was_created())