Docker - unable to run script

Question

What I'm doing

I am using AWS batch to run a docker container for a large compute job. I have configured the ECR/ECS successfully to the best of my knowledge but am having issues running the required commands for reasons that are beyond my level of understanding with docker ( newbie )

What I need to do is pass the below commands into my application and start my application to perform some heavy computing tasks; all commands listed below must be present.

The Issue(s)

The issue arises when I send the submit job to AWS batch; this service pulls the image from the ACR ( amazon container repository ) and spins up a compute environment. The issue comes from when I try to run the command I pass in, below I will go throgh it.

           "command": [
               "mkdir -p logging",
               "chmod 777 logging/",
               "docker run -t -i -e my-application",  # container name
               "-e APIKEY",
               "-e BASEURI",
               "-e APIUSER",
               "-v WORKSPACE /logging:/src/log",
               "DOCKERIMAGE",
               "python my_app.py",
               "-t APP_USER",
               "-e APP_ENVIRONMENT",
               "-u APP_USERNAME",
               "-p APP_PASSWORD",
               "-i IN_PATH",
               "-o OUT_PATH",
               "-b tmp/"
]

The command above generates the following error(s)

container_linux.go:370: starting container process caused: exec: "mkdir -p log": executable file not found in $PATH

I tried to pass in the command to echo the env var $PATH but was unsuccesfull getting a response and resulted in a similar error.

I have ran successfully "ls" and was able to see the directory contents of my application inside.

I am not however able to run any of these commands that I have included in the command [] section. I have tried just running python and such in hopes of getting a more detailed error but was unsuccessful.

Logic in plain English

1. Create a path called logging if it doesnt exist
2. set the permissions for logging
3. run the docker container and pass in the environment variables while doing so
4. Tell docker to run the python file my_app.py and pass in the expected runtime args
5. Execute and perform the required logic deligated in the python3 application

Questions

• Why can I not create a directory here called "logging" where am I ?
• Am I running these properly as defined by AWS batch? or docker
• What am I missing or where am I going wrong?

AWS Batch high level doc

AWS Batch link specific to what i'm doing

Answer 1

Assuming that you're following the syntax described in the Container Properties section of the AWS docs, you have several problems with the syntax of your command directive.

First

The command directive can only run a single command. You can't mash together a bunch of commands as you're trying to do in your example. If you need to run multiple commands you would need to embed them as an argument to a shell. For example, something like:

command: ["/bin/sh", "-c", "mkdir -p logging; chmod 777 logging; ..."]

Second

You must properly tokenize your command lines -- that is, when you type mkdir -p logging at the command prompt, the shell splits this into three parts (or "tokens"): ['mkdir', '-p', 'logging']. You need to do the same thing when building up the list of arguments to command.

This is invalid:

command: ["mkdir -p logging"]

That would looking for a command named mkdir -p logging, and of course no such command exists. That would properly be written as:

command: ["mkdir", "-p", "logging"]

Third

I'm not very familiar with the AWS batch environment, but it's unlikely you can run a docker command inside a docker` container as you're trying to do. It's unclear why you're doing this, though: why not just configure your AWS batch job with the appropriate image, environment variables, etc?

Take a look at some of these example job definitions.