Search code examples
pythonmultithreadingpython-2.7jython

do I need lock to protect mutli-thread race condition in my code


Using Python 2.7 on Windows, and will use Jython which support true multi-threading. The method sendMessage is used to receive message from a specific client, and the client may send the same message to a few other clients (which is what parameter receivers is for, and receivers is a list). The method receiveMessage is used to receive message for a specific client, which are sent from other clients.

The question is whether I need any locks for method sendMessage and receiveMessage? I think there is no need, since even if a client X is receiving its message, it is perfect fine for another client Y to append to message pool to deliver message to client X. And I think for defaultdict/list, append/pop are both atomic and no need for protection.

Please feel free to correct me if I am wrong.

from collections import defaultdict

class Foo:
    def __init__(self):
        # key: receiver client ID, value: message
        self.messagePool = defaultdict(list)
    def sendMessage(self, receivers, message):
        # check valid for receivers
        for r in receivers:
            self.messagePool[r].append(message)
    def receiveMessage(self, clientID):
        result = []
        while len(self.messagePool[clientID]) > 0:
            result.append(self.messagePool[clientID].pop(0))

        return result

Solution

  • I think this question is already well-answered for CPython here and here (basically, you're safe because of GIL, although nothing in documentation (like on defaultdict or list) officially says about that). But I understand your concern about Jython, so let's solve it using some official source, like Jython source code. A pythonic list is a javaish PyList there with this kind of code:

    public void append(PyObject o) {
        list_append(o);
    }
    final synchronized void list_append(PyObject o) {
    ...
    }
    public PyObject pop(int n) {
        return list_pop(n);
    }
    final synchronized PyObject list_pop(int n) {
    ...
    }
    

    And as we have these methods synchronized, we can be sure that list appends and pops are also thread-safe with Jython. Thus, your code seems to be safe wrt threading.

    Although Queue suggestion is still valid one, it really is more appropriate for this use case.