I spin up a dataproc cluster on gcp thru terraform, but I noticed that the component gateway still shows disabled though I had my scripts to enable it.
software_config {
optional_components = [ "ANACONDA", "JUPYTER" ]
image_version = "${var.cluster_version}"
override_properties = {
"dataproc:dataproc.allow.zero.workers" = "true"
"dataproc:dataproc.logging.stackdriver.enable" = "true"
"dataproc:dataproc.enable_component_gateway" = "true"
}
}
While looking up the references, it seems this feature can not be enabled thru terraform and there were also mentions of using endpoint gateway as below,
endpoint_config {
enable_http_port_access = "true"
}
But when I tried using this, It error-ed out as "invalid or unknown key: endpoint_Config" .
Is there any other alternative to get this enabled thru terraform (note: i am using google-beta)
Thank you!
The issue here is that Terraform provider is out of sync with Dataproc API. If you would, please file a feature request on GitHub
For the time being, you may have to create the cluster with CG manually.