Search code examples
rubysocketsparallel-processingchef-infraknife-solo

Why does running "knife solo cook" using Parallel gem give a port error?


We are trying to run a knife solo cook command on remote machines. When we execute the command on each machine sequentially, it works without hiccups. We tried to parallelize this by using the Parallel gem like this:

Parallel.each(machines, in_threads => machines.size) do |machine|
   knife solo cook machine.user + ":" + machine.ip <options>
end

Doing this, the thread that happens to run first executes as expected. However, the other threads do not execute and stop with the error: "EADDRINUSE: Address already in use - bind(2) for 127.0.0.1:8889"

We put in a netstat command at the same place where we call knife solo cook to investigate and found that a Listen socket has been opened on port 8889 by the first thread. We did open any socket, nor have we found any indication that knife or Net:SSH (which is used by knife) are doing this.

Does anyone know who is opening up this socket? Or whether there is any config file for it?

Thanks!

Edit: Forgot to mention, we are doing the same for knife solo prepare before calling cook and that works great. Also, the remote machines all have chef solo set up properly.


Solution

  • Agree with nick, you will have to write a mechanism to listen on multiple ports, or rather spawn ports as per requirement.

    You can also refer the use case of tomcat, we can spawn multiple instances of tomcat for multipe builds, it uses some mechanism to do it. Good use-case to learn about it.