Search code examples
pythonmultithreadingtimer

threading.Timer in Python -- two functions agree on variable identity, but not the value


Are there any caveats similar to variable caching when using threading.Timer in Python?

I'm observing an effect similar to not putting "volatile" keyword in other languages, but I've read that this doesn't apply to Python and yet something is off. In summary, two methods (threads?) agree on the identity of a list variable but disagree on the contents.

I have a class with member variable self.x (a list) which is assigned once in constructor and then its identity never changes (but it can be cleared and refilled).

A Timer is also started in the constructor that periodically updates the contents of self.x. I'm not using any locks though I probably should (eventually).

Other users of the class instance then sometimes try to read the contents of the list.

Problem: At the end of the timer handler, the list is populated correctly (I've printed its contents in the logs) but when a user of the class instance reads the instance variable, it's empty! (more specifically, same value as it was initialized in the constructor, i.e. if I put some other item in there, it'll be that value).

The weird thing is that the getter and the timer agree on the identity of the list! (id(self.x) returns the same value). Also, I was not able to repro this in tests, even though I'm doing the same thing.

Any idea what I might be doing wrong?

Thanks in advance!


Solution

  • Seems like I misunderstood how multiprocessing works. What happened is the code forked into multiple processes and even though the id() of objects remained the same, I didn't realize they were in different (forked) processes. I knew this was happening but I thought that the fork would take care of copying the timer as well. I've changed the code to create the object (and timers) once the fork is complete and it seems to have solved the problem.