Authenticate Google Composer http call task with IAP protected app

Question

I have a setup where I have an app engine REST application and a Google composer / airflow DAG that has a task where it is supposed to fetch data from one of the endpoints of the app. The app is protected by IAP. I have added the service account under which Airflow runs to the "IAP-secured Web App User" list, however each time the step executes the response to the http call is the Google Sign-In page. Any idea if any additional step is needed?

The code for my DAG step:

def get_data():
    r = requests.get(url="url-to-my-app-endpoint>")
    print('stuff:')
    print(r.status_code)
    print(r.content)
    return 1

# ...

python_fetch_data = PythonOperator(
    task_id='python_fetch_data',
    python_callable=get_data,
    dag=dag,
    depends_on_past=True,
    priority_weight=2
)

Answer 1

https://cloud.google.com/iap/docs/authentication-howto#authenticating_from_a_service_account explains how to extend your DAG code so that it sends credentials to the IAP-protected API backend.

A bit of background: Since Composer is built on top of GCP, your Composer deployment has a unique service account identity that it's running as. You can add that service account to the IAP access list for your endpoint.

I don't know if the Composer UI makes it easy to see the "email" address for your service account, but if you add the code above and decode the token it generates, that will show it.