Search code examples
ramazon-web-servicesopencpusnow

How to setup AWS cluster to work with openCPU?

I have two EC2 machines: master and slave. SSH keys are generated for user ubuntu and saved to ~/.ssh/authorized_keys on both machines. Thus I can use the cluster from master node as ubuntu user like this:

library(doSNOW)

cluster_options <- 
c(rep(list(
        list(host = "ec2-xx-xx-xx-xx.compute-1.amazonaws.com",
             snowlib = "/usr/local/lib/R/site-library")), 2))
cl <- makeCluster(cluster_options, type = "SOCK")
clusterApply(cl, 1:2, get("+"), 3)
stopCluster(cl)

But when I call it via openCPU it gives permission denied message.

Currently I'm thinking about two possible solutions:

  1. Add SSH keys for opencpu user. But I don't have idea how to do it as opencpu is non-interactive user
  2. Make slaves accessible by master without any SSH keys

I'd prefer the first way and definitely need help here. But second way is also ok.

Solution

  • Finally I ended up with solution. It has several aspects:

    1. Hostbased Authentication should be configured between two EC2 nodes. Good tutorial can be found here: https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Host-based_Authentication
    2. OpenCPU should be installed on both nodes.
    3. SSH keys should be generated for www-data user (R process is executed with this user). Delicate aspect here is that www-data is non-interactive user, so we need to make it interactive (edit /etc/passwd), generate SSH keypair for www-data, add public key to server node and make www-data non-interactive again.

    Not so elegant, but it works :)