GCP - retrieve cloud composer proxy url - throws error Expected OAuth 2 access toke

Question

I am trying to get the cloud composer proxy url to trigger dags from cloud function. I am following below blog. however it's failing while retrieving this url.

https://medium.com/google-cloud/calling-cloud-composer-to-cloud-functions-and-back-again-securely-8e65d783acce

Command

curl -s  -H "Authorization: Bearer `gcloud auth application-default print-access-token`" https://composer.googleapis.com/v1/projects/test/locations/us-east1/environments/test-environment

I tried gcloud auth print-access-token but same error

error

{
  "error": {
    "code": 401,
    "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
    "status": "UNAUTHENTICATED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "CREDENTIALS_MISSING",
        "domain": "googleapis.com",
        "metadata": {
          "method": "google.cloud.orchestration.airflow.service.v1.Environments.GetEnvironment",
          "service": "composer.googleapis.com"
        }
      }
    ]
  }
}

Answer 1

I followed the medium post that you shared. NOTE (where my_project_id is, you have to set your project id).

First I created the project (My user has an owner role in the project) and the environment.

export GOOGLE_PROJECT_ID=`gcloud config get-value core/project`

export PROJECT_NUMBER=`gcloud projects describe my_project_id --format='value(projectNumber)'

gcloud composer environments create composer1  --location us-central1

gcloud composer environments  list --locations us-central1

After these steps I didn’t configure the requirements.txt. But I configured the connection.

gcloud composer environments update composer1  \
 --update-env-variables=AIRFLOW_CONN_MY_GCF_CONN=https://us-central1-my_project_id.cloudfunctions.net --location us-central1

gcloud composer environments describe composer1 --location us-central1

gcloud composer environments describe composer1 --location us-central1 --format="get(config.dagGcsPrefix)"

Then I went to Composer and I set the python packages manually.

You have to click on EDIT and add the name of the packages.

After the packages are installed you need to know if you have access to your token. Here is some documentation on how to create the a service account. Then you need to set your credential in the terminal,command prompt or shell that you are using. And here is some documentation on how to obtain the access token. You need to have a Service Account to have the token.

Then I follow step 3 where it brings you an error message.

curl -s  -H "Authorization: Bearer `gcloud auth print-access-token`" https://composer.googleapis.com/v1beta1/projects/my_project_id/locations/us-central1/environments/composer1 | jq [.config.airflowUri]

And this brings me, my URL for airflow that look like:

[

"https://url-tp.appspot.com"

]