I am trying to trigger a GCP Cloud Run job from a python script following the run_job documentation (https://cloud.google.com/python/docs/reference/run/latest/google.cloud.run_v2.services.jobs.JobsClient#google_cloud_run_v2_services_jobs_JobsClient_run_job). However, I'm getting errors that I haven't been able to debug.
This is a job that already exists, but I need to overwrite an environment variable.
Here's is my code:
env_vars = [
run_v2.EnvVar(name="VAR_1", value="var_1_value"),
run_v2.EnvVar(name="VAR_2", value="var_2_value"),
]
# Set the env vars as container overrides
container_overrides = run_v2.RunJobRequest.Overrides.ContainerOverride(
name="myjobname",
env=env_vars
)
request_override = run_v2.RunJobRequest.Overrides(
container_overrides=container_overrides
)
# Initialize the request
job_name = f"projects/myproject/locations/mylocation/jobs/myjob"
request = run_v2.RunJobRequest(
name=job_name,
overrides=request_override
)
# Make the request
operation = client.run_job(request=request)
logging.info("Waiting for operation to complete...")
response = operation.result()
logging.info(f"Operation result: {response}")
And this is the error I'm getting:
Traceback (most recent call last):
File "/opt/python3.8/lib/python3.8/site-packages/airflow/decorators/base.py", line 220, in execute
return_value = super().execute(context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/operators/python.py", line 181, in execute
return_value = self.execute_callable()
File "/opt/python3.8/lib/python3.8/site-packages/airflow/operators/python.py", line 198, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/home/airflow/gcs/dags/etl.py", line 128, in run_acolite
request = run_v2.RunJobRequest(
File "/opt/python3.8/lib/python3.8/site-packages/proto/message.py", line 604, in __init__
super().__setattr__("_pb", self._meta.pb(**params))
TypeError: Message must be initialized with a dict: google.cloud.run.v2.RunJobRequest
Thank you!
We recently started using the Cloud Run Jobs service in my workplace and I found myself needing to carry out the same task.
A dictionary with the Override Specification is required. I've amended the Initialize request block as per your example.
override_spec = {
'container_overrides': [
{
'env': [
{'name': 'VAR_1', 'value': 'var_1_value'}
]
}
]
}
# Initialize the request
job_name = f"projects/myproject/locations/mylocation/jobs/myjob"
request = run_v2.RunJobRequest(
name=job_name,
overrides=override_spec
)