Search code examples
pythonnode.jsposixipcmqueue

How to IPC between node and python using posix message queue?

I'm trying to write a "hello world" of IPC between a node js and a python3 application. At the moment I'm having a parse error when a new message arrives the Python app. The codes are:

Python:

from ipcqueue import posixmq
from time import sleep

q1 = posixmq.Queue('/fila1')

while True:

    while q1.qsize() > 0:
        print('p1: Recebi na minha fila: ' + str(q1.get()))

    sleep(0.5)

Node:

const PosixMQ = require('posix-mq')

var mq = new PosixMQ();
mq.open({
    name: '/fila1',
    create: true,
    mode: '0777',
    maxmsgs: 10,
    msgsize: 11
});
mq.push("hello world")
mq.close();

When the second app sends the message, the python app fails with:

File "../test-fila1.py", line 9, in 

print('p1: Recebi na minha fila: ' + str(q1.get()))   File "/usr/lib/python3.7/site-packages/ipcqueue/posixmq.py", line 174, in

get

return self._serializer.loads(data)   File "/usr/lib/python3.7/site-packages/ipcqueue/serializers.py", line 14,

in loads

return pickle.loads(data) KeyError: 101

[2]- Exit 1 python3 ../test-fila1.py [3]+ Done node index.js

EDIT
So, I decided to change the question from "KeyError: 101 when loading ipc message between node and python apps" to "How to IPC between node and python using posix message queue?" because I just noticed that the sample code I posted generates different errors everytime I run them. Now the error that is happening is (however, no code was changed):

Traceback (most recent call last): File "snippets/test-fila1.py",
line 9, in
print('p1: Recebi na minha fila: ' + str(q1.get())) File "/usr/lib/python3.7/site-packages/ipcqueue/posixmq.py", line 174, in
get
return self._serializer.loads(data) File "/usr/lib/python3.7/site-packages/ipcqueue/serializers.py", line 14,
in loads
return pickle.loads(data)
_pickle.UnpicklingError: unpickling stack underflow

Solution

  • If you look the __init__ of the Queue

    class Queue(object):
    """
    POSIX message queue.
    """

    def __init__(self, name, maxsize=10, maxmsgsize=1024, serializer=PickleSerializer):

    As you can see the default serializer is PickleSerializer, which means it assumes the data coming on the queue is pickled, while you are sending raw data from nodejs. The fix is simple, use the RawSerializer

    from ipcqueue import posixmq
from time import sleep
from ipcqueue.serializers import RawSerializer
q1 = posixmq.Queue('/fila1', serializer=RawSerializer)

while True:

    while q1.qsize() > 0:
        print('p1: Recebi na minha fila: ' + str(q1.get()))

    sleep(0.5)

    Working fine