Failing to connect via SSH to EC2 from Bitbucket pipeline

Question

I have a Bitbucket pipeline set up that looks like this (previous steps, which are about building and pushing the docker image, are omitted but work):

      - step:
          name: Deploy to EC2
          script:
            - |
              ssh -i ${EC2_KEY_PEM} ${EC2_USER}@${EC2_HOST} <<'EOF'
              docker ps -q | grep -q . && docker stop $(docker ps -q)
              echo ${DOCKER_PASSWORD} | docker login --username ${DOCKER_USERNAME} --password-stdin
              docker pull ${DOCKER_REPO}/${IMAGE_NAME}:${IMAGE_VERSION}
              docker run -d --name my_container_name ${DOCKER_REPO}/${IMAGE_NAME}:${IMAGE_VERSION} 
              EOF
          services:
              - docker

where ${EC2_KEY_PEM} is a repo variable containing the original .pem file.

My Bitbucket pipeline keeps failing on the following error:

<1s
+ ssh -i ${EC2_KEY_PEM} ${EC2_USER}@${EC2_HOST} <<'EOF'
docker ps -q | grep -q . && docker stop $(docker ps -q)
echo ${DOCKER_PASSWORD} | docker login --username ${DOCKER_USERNAME} --password-stdin
docker pull ${DOCKER_REPO}/${IMAGE_NAME}:${IMAGE_VERSION}
docker run -d --name my_container_name ${DOCKER_REPO}/${IMAGE_NAME}:${IMAGE_VERSION} 
EOF
Warning: Identity file @ not accessible: No such file or directory.
usage: ssh [-46AaCfGgKkMNnqsTtVvXxYy] [-B bind_interface]
           [-b bind_address] [-c cipher_spec] [-D [bind_address:]port]
           [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11]
           [-i identity_file] [-J [user@]host[:port]] [-L address]
           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]
           [-Q query_option] [-R address] [-S ctl_path] [-W host:port]
           [-w local_tun[:remote_tun]] destination [command]

(I have tried with <<EOF instead of <<'EOF')

I have also tried the following for defining the private ssh key as follows:

            - echo ${EC2_KEY_BASE64} | base64 -d > ec2-key.pem
            - chmod 600 ec2-key.pem
            - |
              ssh -i ec2-key.pem ${EC2_USER}@${EC2_HOST} <<'EOF'

where ${EC2_KEY_BASE64} is converted from .pem to a base64 string. Same error in all instances.

Answer 1

The issue was the location of the ${EC2_KEY_BASE64} variable. Bitbucket has Repository variables and Deployment variables. What is not shown in my bitbucket-pipelines.yml, is that I marked a previous step as deployment: Production, assuming this will roll through to the next step.

It does not. In other words the command echo ${EC2_KEY_BASE64} | base64 -d > ec2-key.pem produced an empty ec2-key.pem file resulting in the subsequent failure.

Key takeaways:

1. A specific deployment (eg Production) can only be used in one step.
2. If you have values defined as Production deployment variables in a specific step in your bitbucket-pipelines.yml, they will not be available in any other step.