Search code examples
pythontornado

Python Dictionaries and Threading Concurrency


I'm using a non-blocking asynchronous threading websockets protocol (Tornado) in python 3. I'm using a dictionary very conservatively in the global level, but it can be updated by any number of threads (basically when a new user connects, their socket is mapped to an object via the dictionary). I've made a basic system to deal with concurrency... but it's not any official method. Since the sockets are unique, I don't think I should have any problem with people trying to assign values to the same socket at the same time, but I was wondering if with scale, there might be another problem, such as trying to get something from the dictionary when it's being resized. For reference, here's the "concurrency-fix" I came up with.

class Routing_Table:

    def set(self, key, value):
        self.blocked = self.blocked + 1
        self.table[key] = value
        self.blocked = self.blocked - 1

    def get(self, key):
        while(self.blocked > 0):
            pass
        return self.table[key]

    def __init__(self):
        self.table = {}
        self.blocked = 0

EDIT: Also, do you think I should add a method similar to set for deleting entries?


Solution

  • If you want to do anything in a thread-safe manner, the basic idea is:

    class ConcurrentThingy:
        def __init__ (self):
            self.lock = Lock () # or RLock () if you want it to be reentrant    
    
        def concurrentlyAccessedMethod (self, *args, **kwargs):
            with self.lock: doMeanStuff ()
    

    Your class could look something like this:

    class Routing_Table:
        def set (self, key, value):
            with self.lock: self.table[key] = value
    
        def get(self, key):
            with self.lock: return self.table[key]
    
        def __init__(self):
            self.table = {}
            self.lock = Lock ()