END GOAL: Build a flask app that can ssh into a remote machine that then remotes to a remote device (via agent forwarding) and fetches the value of a config file on that device.
PROBLEM: I have a script that works when I run the flask app on my local machine. It fetches and displays the values of the config from the remote device. But when I build a docker container to host the flask app, I only ever receive a Permission denied (publickey). error from the api. I have the docker container set up in a way that I am able to execute the exact same ssh script to fetch the config from the container's terminal. But even if I curl the api it gives me the same permission denied error.
This is my little flask api and script to fetch the contents of the config.
from flask import Flask, jsonify, request
app = Flask(__name__)
@app.get('/api/v1/remote/config')
def ssh_show_config():
user = request.args.get('user')
host = request.args.get('host')
node = request.args.get('node')
return jsonify(get_config_via_ssh(user, host, node))
def get_config_via_ssh(user, host, tunnel_ip):
try:
cmd2 = f"ssh -A -tt {user}@{host} -tt \"ssh root@{tunnel_ip} 'cat /storage/config'\""
process = subprocess.getoutput(cmd2, encoding='utf-8')
# ConfigCleaner is just a tool that cleans the data in the config
cleaner = ConfigCleaner()
return cleaner.clean(process)
except Exception as e:
print(e)
print("[!] Cannot connect to the device SSH Server")
exit()
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
This is my Dockerfile
FROM python:3.11-slim-bullseye
# update/install necessary alpine linux libraries
RUN apt-get -y update
RUN apt-get install -y openssh-server
RUN apt-get install -y openssh-client
RUN apt-get install -y curl
RUN apt-get install -y sudo
RUN adduser tester
RUN adduser tester sudo
RUN pip3 install poetry
# create project directory
WORKDIR /root/config-app
COPY . .
EXPOSE 5000
EXPOSE 22
# install required python libraries via poetry
RUN poetry config virtualenvs.create false
RUN poetry install
USER tester
# switch to app directory and run app
WORKDIR /root/config-app/src/config_fetcher
CMD poetry run flask --app app run --host 0.0.0.0 --port 5000
This is my docker-compose
version: '3.8'
services:
config-fetcher:
image: config-app
user: root
volumes:
- ~/.ssh/config:/root/.ssh/config:rw
- ~/.ssh/id_ed25519:/root/.ssh/id_ed25519:rw
- ~/.ssh/id_ed25519.pub:/root/.ssh/id_ed25519.pub:rw
build:
context: .
dockerfile: ./Dockerfile
ports:
- "5000:5000"
- "22:22"
After I run docker compose up, I execute the following command in the container's terminal in order to 1. add a blank known_hosts file and a sockets directory and 2. start the ssh agent and add my mounted key to it in the container:
touch ~/.ssh/known_hosts && mkdir ~/.ssh/sockets && eval ssh-agent && ssh-add ~/.ssh/id_ed25519
After all of that, I am able to execute
ssh -A -tt user@host.domain.com -tt ssh root@device.ip cat /storage/config
inside the container's terminal and it will successfully return the content of the config file (just as I've designed the flask api to do). However, if I curl the api endpoint I created for my flask app (from within the same container), it gives me the permission error. None of this is a problem if I run the flask app on my local machine without the use of docker.
I'm really beating myself over the head on this one.
There are a bunch of problems here.
Your use of ssh-agent is ineffective.
Because you're only starting ssh-agent as part of your interactive docker exec session, the agent isn't going to be visible to the ssh process spawned by your flask application.
I'm suspicious of your ssh command line, because you've got three levels of quoting going on.
By ditching getoutput in favor of check_output and specifying the command as a list instead of a string, we can simplify things.
We can further simplify things by using the -J option to ssh to specify a jump host.
EXPOSE in a Dockerfile does nothing (it's just informative).
We can fix your code by writing it like this:
import subprocess
from flask import Flask, jsonify, request
app = Flask(__name__)
@app.get("/api/v1/remote/config")
def ssh_show_config():
user = request.args.get("user")
host = request.args.get("host")
node = request.args.get("node")
return jsonify(get_config_via_ssh(user, host, node))
def get_config_via_ssh(user, host, tunnel_ip):
try:
cmd = [
"ssh",
"-A",
"-J", f"{user}@{host}",
f"root@{tunnel_ip}",
"cat /tmp/config",
]
config = subprocess.check_output(cmd, encoding="utf-8", stderr=subprocess.PIPE)
return {"config": config}
exception Exception as e:
return {"error": f"Cannot connect to the device SSH Server: {e}"}
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)
I've rewritten the Dockerfile using pipenv instead of poetry (because I'm not familiar with poetry) and removed a bunch of extraneous code that wasn't relevant to this example.
I've also modified the Dockerfile to run your application under the control of ssh-agent, with an explicit agent socket so that we can interact with the same agent from an interactive session:
FROM python:3.11-slim-bullseye
# update/install necessary alpine linux libraries
RUN apt-get -y update
RUN apt-get install -y openssh-client
RUN pip install pipenv
# create project directory
WORKDIR /app
COPY requirements.txt ./
RUN pipenv install -r requirements.txt
COPY . ./
RUN mkdir -m 700 -p /root/.ssh && cp ssh_config /root/.ssh/config && chmod 600 /root/.ssh/config
ENV SSH_AUTH_SOCK=/tmp/agent.sock
CMD ["ssh-agent", "-a", "/tmp/agent.sock", "pipenv", "run", "flask", \
"--app", "app", "run", "--host", "0.0.0.0", "--port", "5000"]
Lastly, I've updated the compose.yaml to mount an ssh key as a docker secret:
services:
config-fetcher:
image: flaskapp
user: root
build:
context: .
ports:
- "5000:5000"
secrets:
- sshkey
secrets:
sshkey:
file: ~/.ssh/id_rsa
This means that when the container runs, my ssh key will be available as /run/secrets/sshkey.
With all of this in place, if I docker compose up the container, I can then start an interactive session and load the key:
$ docker compose exec config-fetcher ssh-add /run/secrets/sshkey
Enter passphrase for /run/secrets/sshkey:
Identity added: /run/secrets/sshkey (/run/secrets/sshkey)
And then the following request works:
$ curl localhost:5000/api/v1/remote/config'?user=lars&host=172.20.0.1&node=127.0.0.1'
{"config":"this is a test\n"}