Search code examples
google-cloud-dataprocgoogle-iam

how to create dataproc cluster by service account


I am quite confused by this document enter link description here

Service account requirements and Limitations:
* Service accounts can only be set when a cluster is created.
* You need to create a service account before creating the Cloud Dataproc cluster that will be associated with the service account.
* Once set, the service account used for a cluster cannot be changed.

Dose this means I cannot create a service account, which have role to create a dataproc cluster? For Now, I can only create a dataproc cluster by my own account " gcloud auth login" But I want create dataproc cluster from jenkins by setup

gcloud auth activate-service-account --key-file


Solution

  • Yes, you can use a service account to create Dataproc clusters and submit jobs. However, the link you refer deals with running Dataproc clusters with a service account which isn't applicable to your concern.

    To create a Dataproc using a service account:

    1. Create a service account

    2. Assign Cloud Dataproc Editor role

    3. Download its json credentials file

    4. Configure authentication mechanism:

      4.1 gcloud auth activate-service-account --key-file=JSON_FILE_PATH

      4.2 GOOGLE_APPLICATION_CREDENTIALS=JSON_FILE_PATH

    5. Create your Dataproc cluster