Connect Node.js as client to a Common Lisp server

I've got small but CPU heavy app in alpha stage in node.js, it's a small game. I'm running into performance issues and I need to speed it up by at least a factor of 20 to get to beta. And since parallel execution would get me very far, I decided that good start would be to share the game map between processes or threads that would perform parallel operations on it. That's pretty impossible to do in node, so I decided to write the meaty parts in CL (SBCL + Linux) and connect to it through unix domain socket.

The plan is:

[players] <-> (node.js front server) <-> (SBCL performing game ticks)

The point is, I need to pass fast messages between node.js and SBCL in a matter similar to socket.io.

Here is what didn't work (you can skip this part)

On Node side, I can't use plain socket.io because it doesn't support Unix Domain Sockets, but net module does, So I can at least do socket.write('raw data') - better than nothing for now.

On CL side, I tried to run woo web server (it supports local sockets) and I could connect to it from node and pass raw data around, but there are all the unnecessary HTTP parts involved and woo is always running as server; it's waiting for GET / HTTP/1.1 ..... I didn't find a way to actually initiate a message from woo first. Also, it's totally undocumented and uncommented and involves lot of FF calls to C libs, which I'm not at all familiar with.

So I went through several more CL web servers that didn't compile, didn't support unix sockets, were abandoned or undocumented, eventually moved to plain sb-bsd-sockets and finally to iolib, but I still can't figure it out.

iolib looked promising, but I can't connect to it from node.

I've got this:

(with-open-socket (socket :address-family :local
                          :type :datagram
                          ;; :connect  :passive
                          :local-filename "/tmp/socket")

  (format t "Socket created")
  ;; (listen-on socket)
  ;; (bind-address socket (make-address "/tmp/socket"))
  (loop
     (let ((msg (receive-from socket :size 20)))
       (format t msg))))

and I'm getting

#<Syscall "recvfrom" signalled error EWOULDBLOCK(11) "Resource temporarily unavailable" FD=6>
   [Condition of type IOLIB/SYSCALLS:EWOULDBLOCK]

Restarts:
 0: [IGNORE-SYSCALL-ERROR] Ignore this socket condition
 1: [RETRY-SYSCALL] Try to receive data again
 2: [RETRY] Retry SLIME interactive evaluation request.
 3: [*ABORT] Return to SLIME's top level.
 4: [ABORT] abort thread (#<THREAD "worker" RUNNING {10055169A3}>)

I don't know if I should call something like accept-connection or listen-to on that socket first. All I tried resulted in errors. Also, if I [RETRY-SYSCALL] in repl, the error goes away for about 10 seconds but comes back. In this time, node still can't connect.

This seems to get more complicated than I thought. I've already lost ~6 hours of work on iolib alone and I didn't even start on parsing the messages, learning how to create events from them, converting between JSON and s-exps etc..

My questions are:

how do i set this connection up in iolib so that node's net can connect?
Assuming I can choose, what type of connection would be best suited for passing events/messages? (datagram / stream)
Are there some working tools that I didn't try?
Also, are there some other libs than iolib that are perhaps more high-level / better documented?
Are there any better/easier/faster approaches to this performance / concurrency problem?
Any other ideas?

I'm close to just ditching the idea of CL and use something like in-memory mongo with several node processes instead (..it doesn't really sound fast) but I love lisp, it would be great to have things like lparallel on the backend. I just haven't moved an inch since yesterday morning, I just can't figure out the libs. Perhaps I should learn clojure instead.

PS: I wouldn't normally ask for "write me teh code", but if some good soul is around, I would really appreciate it, even in pseudocode.

PPS: Any radically different approaches are also welcome. Please, speak up your mind :)

Thanks for reading!

Solution

So in the end, I figured it out...

(with-open-socket  (socket :address-family :local
                           :type :stream
                           :connect  :passive
                           :local-filename "/tmp/node-sbcl.sock")

  (log-message :info "Waiting for client...")
  (setf *client* (accept-connection socket :wait t))
  (log-message :info "Accepted client's connection.")

  ;; ...lunch with *client* + the bits for parsing json and separating messages...
  )

I switched to :type :stream and most problems disappeared. accept-connection has to be called on socket, but listen-to must not. I had to write a way to separate messages myself, but it was lot easier than I thought. For some reason, :type :datagram just didn't work, I don't know why.

And in node:

var JsonSocket = require('json-socket');
var net = require('net');
var sbcl = new JsonSocket(net.Socket());


sbcl.connect("/tmp/node-sbcl.sock");
sbcl.on('connect', function(){
    console.log('Connected to SBCL, YAY!');

    console.log('Sending hi!');

    sbcl.sendMessage({'cmd': "heyThere"});

    sbcl.on('message', function(message){
        if(!message.cmd) {
            console.log("We've received msg from SBCL with no CMD!!!'");
            return;
        }

        switch(message.cmd){
        case 'heyNode': console.log('SBCL says hi...'); break;
        }
    });
});

So this works, in case somebody else has some similar chicken ideas of using lisp and node together.