I have python script who's responsibility is to run docker image. For running the docker image I am using docker client and the run command is defined like this -
def run_docker(self, cred, image, env, volumes, cmd):
"""
Docker image needs to be executed after image is successfully pulled,
and docker command is contructed based on the job parameters.
"""
self.log.info('run_docker', state=self.state)
self._docker_login(cred)
self.docker_client = docker.from_env()
self.container = self.docker_client.containers.run(
image,
environment=env,
volumes=volumes,
command=cmd,
stderr=True,
remove=True,
)
There are two problems - 1) When you remove the container the docker log file which captures the STDOUT and STDERR together is also removed. Because of this issue, I decided to write STDOUT and STDERR to two different files, so that the user gets to analyse STDERR and STDOUT with maximum flexibility.
While using the docker client as mentioned here - https://docker-py.readthedocs.io/en/stable/containers.html The parameters have been modified accordingly in the run command. But I am unable figure out how to accomplish this. Also docker client doesn't mention how to use logging plugins.
Any help or suggestions are welcome.
Don't use remove=True. You can call self.container.remove() later.
I would call run() with detach=True like this:
self.container = self.docker_client.containers.run(
image,
environment=env,
volumes=volumes,
command=cmd,
stderr=True,
detach=True,
)
Then I would wait with self.container.wait() and retrieve the logs for stdout & stderr separately like this:
out = self.container.logs(stdout=True, stderr=False)
err = self.container.logs(stdout=False, stderr=True)