Creating Dataproc Cluster with public-ip-address using DataprocCreateClusterOperator in Airflow...

Is there a way to pass the dataproc version via the Airflow DataprocCreateBatchOperator method?...

Error while scanning intermediate done dir - dataproc spark job...

Set Spark configuration when running python in dbt for BigQuery...

what's difference between dataproc cluster on GKE vs Compute engine?...

Trigger spark submit jobs from airflow on Dataproc Cluster without SSH...

Not able to write into BigQuery JSON Field with Pyspark...

How to use --properties-file flag in dataproc?...

Pub/Sub Publish message from Dataproc cluster using Python: ACCESS_TOKEN_SCOPE_INSUFFICIENT...

Configure trino-jvm properties in GCP Dataproc on cluster create...

Dataproc Workflow(ephemeral cluster) or Dataproc Serverless for batch processing?...

In Dataproc, whether or not the file prefix should be used when applying a property to job?...

Pyspark with custom container on GCP Dataproc Serverless : access to class in custom container image...

How to enable outside connection before submit Pyspark job to Dataproc...

Yarn CPU usage and the result of htop on workers are incosistent. I am running a SPARK cluster on Da...

Yarn allocates only 1 core per container. Running spark on yarn...

Packaging PySpark with PEX environment on dataproc...

GCP Dataproc Base Docker Image...

How to use new Spark Context...

Couldn't connect to dpms while creating dataproc using airflow operator...

How to enable Spark web interface on Dataproc(GCP) using DataprocCreateClusterOperator of Apache Air...

GCP Dataproc - adding multiple packages(kafka, mongodb) while submitting jobs not working...

DataprocClusterCreateOperator doesnt have temp_bucket variable to define...

Why is my hdfs capacity not remainng constant?...

Where GCP dataproc stores notebook instances?...

Why is there just 1 job id in dataproc when there are multiple actions in the pyspark script?...

PySpark runs in YARN client mode but fails in cluster mode for "User did not initialize spark c...

Where to find spark log in dataproc when running job on cluster mode...

Connect PySpark session to DataProc...

How are spark jobs submitted in cluster mode?...