GC Doesn't Delete Circular References in WeakKeyDictionaries?

Question

I have a situation in which I'd like to maintain a mapping from one object to another for as long as the first object exists. My first thought was to use a WeakKeyDictionary.

import weakref
import gc

class M:
    pass

w = weakref.WeakKeyDictionary()
m = M()
w[m] = some_other_object
del m
gc.collect()
print w.keys()

This works fine in most cases. However, in the case where some_other_object is (or has a reference to) m, the M instance will not be garbage collected. (To see an example, replace some_other_object with m)

Of course, if I stored the mapping on the object itself, it would be garbage collected when I deleted it:

import weakref
import gc

class M:
    pass

m = M()
m.circular_reference = m
r = weakref.ref(m)
del m
gc.collect()
print r

Can I achieve the results of the second example using the weakref module (i.e. without mutating m)?

In other words, can I use the weakref module to map an object to itself (or another object that has a strong reference to it) and only keep the object in memory as long as there are other references to it?

Answer 1

You don't really have circular references in these examples. Your circle goes like:

WeakKeyDict -> value -> key -> ...

So the dict keeps the value alive, which in turn keeps the key alive, and through a separate mechanism that does not involve a strong reference, the key instructs the dict to keep the value alive. Since there is no proper reference circle here, the GC never detects any issue. (However, your second example does contain a circular reference, which is why it behaves like the first)

I suspect the only solution to your problem is to ensure that the values of the dict never have strong references (direct or indirect) to any of the keys in the dict. This is similar to what srgerg proposed, but rather than making weak references to all values, you really want references to the keys to be weak. For example:

import weakref
import gc

class M:
    pass

w = weakref.WeakKeyDictionary()
key = M()
val = M()
val.keyRef = weakref.ref(val)
w[key] = val
print len(w)   ## prints 1
del key
gc.collect()
print len(w)   ## prints 0

This way, there are always strong references to the values, but you are carefully controlling references to keys to determine what gets removed from dict. Depending on the complexity of your program, this could be time consuming to implement, since you need to manually track down all references to keys.

But perhaps we can propose a simpler solution if you tell us more about the specific problem..