what's difference between dataproc cluster on GKE vs Compute engine?

Question

We can now create dataproc clusters using compute engine or GKE. What are the major advantages of creating a cluster on GKE vs Compute Engine. We have faced problem of insufficient resources in zone error multiple times while creating cluster on compute engine. Will it solve this issue if we use GKE for cluster and what are the cost difference between them.

Answer 1

To solve the error message “insufficient resources in zone”, You may refer to this GCP documentation.

To answer your question, what's the difference between dataproc cluster on GKE vs GCE.

In GKE, You can directly create a Dataproc cluster and do your deployments.

You may also check the advantages of GKE in documentation or check the GKE features below:

1. Run your apps on a fully managed Kubernetes cluster with GKE autopilot.
2. Start quickly with single-click clusters and scale up to 15000 nodes
3. Leverage a high-availability control plane including multi-zonal and regional clusters
4. Eliminate operational overhead with industry-first four-way auto scaling
5. Secure by default, including vulnerability scanning of container images and data encryption

While in GCE, you have to manually install Kubernetes, then create your Dataproc cluster and do your deployments.