I am having a problem running a Cloud function I created using Python on the Google Cloud Platform. I am attempting to trigger a DAG using the cloud function and I follow all the rules specified in the official documentation to do it. I keep getting this error code.
EDT
gcs-dag-trigger-function
68hsnlvog0g6
Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
response = self.full_dispatch_request()
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1516, in full_dispatch_request
rv = self.dispatch_request()
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1502, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/__init__.py", line 171, in view_func
function(data, context)
File "/workspace/main.py", line 48, in trigger_dag
make_iap_request(
File "/workspace/main.py", line 87, in make_iap_request
raise Exception(
Exception: Bad response from application: 404 / {'Date': 'Fri, 15 Jul 2022 16:25:22 GMT', 'Content-Type': 'text/html; charset=utf-8', 'Vary': 'Accept-Encoding', 'Server': 'gunicorn', 'X-Robots-Tag': 'noindex, nofollow', 'Set-Cookie': 'session=d8fb37f6-bd2d-4a2c-8abc-e528eadc9ac3.hUuadQnCRiJzbuufKObZIMtsTL8; Expires=Sun, 14-Aug-2022 16:25:22 GMT; HttpOnly; Path=/; SameSite=Lax', 'Content-Encoding': 'gzip', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'} / '\n\n<!DOCTYPE html>\n<html lang="en">\n <head>\n <title>Airflow 404</title>\n <link rel="icon" type="image/png" href="/static/pin_32.png">\n </head>\n <body>\n <div style="font-family: verdana; text-align: center; margin-top: 200px;">\n <img src="/static/pin_100.png" width="50px" alt="pin-logo" />\n <h1>Airflow 404</h1>\n <p>Page cannot be found.</p>\n <a href="/">Return to the main page</a>\n <p>3405c1ef5ce7</p>\n </div>\n </body>\n</html>'
Can anyone help me with this?
@AnjelaB it turns out because I am using Airflow 2 (this is the version which launches in Cloud Composer), you have to use the code, USE_EXPERIMENTAL_API=False in your cloud function since you are now using the stable API. Secondly, your regular service account won't work because its characters are too long (i.e. >64) & you can only use a SA with a limited number of characters. The work around is to manually add a SA as an Airflow user using the documentation found at this link