I created a docker container with a python script to execute using the Cloud Run Job from GCP. My script runs a code like this:
python3.9 extract_parquet.py --table_name users
I used the Google Run cli to create my job, running the following command:
gcloud run jobs create extract_parquet --region=us-central1 --memory=4Gi --cpu=4 --task-timeout=4h --max-retries=0 --image=us-east1-docker.pkg.dev/my-project/hyper-suite/hyper-suite:latest --command="python3.9, extract_parquet.py, --table_name, users"
However, when I try to execute this job, I get the following error:
/usr/bin/python3.9: can't open file '/usr/src/app/ extract_parquet.py': [Errno 2] No such file or directory
Note that there is an extra whitespace after /app/ which indicates the problem, although I don't know what this behavior occurs. If I manually edit the command option on Google interface and hit the backspace button, the command runs smoothly.
I also found a possible problem on the yaml configuration as showed below. There is no apostrophe on python command, but I'm not sure if this impact on my problem.
UPDATE: manually edit the command option on Google interface the yaml removes the apostrophes from my execution command as shown below:
Anyone knows why this is happening and how to solve this? Sincerely.
The issue comes from the command line. Remove the space here
Don't do that
--command="python3.9, extract_parquet.py, --table_name, users"
But do this
--command="python3.9,extract_parquet.py,--table_name,users"
space after a comma is for human readability. Here, the comma is a arg separator, no human involved!