How to disable logs for whole cluster in gce
Could it be possible for already created (Yarn/Hadoop) cluster to disable logging for all servers inside ?
I can't find anything like it. Is there anything in Dataproc or Compute Engine which can help me to disable the logs ?
With little help and suggestion from Google support, there is complete solution to skip logging for whole yarn/hadoop cluster.
That can be possible only when create new cluster from dataproc either by google cloud page or console.
Property which should to be set in cluster properties filed:
dataproc:dataproc.logging.stackdriver.enable to be false
More info at: https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/cluster-properties
If you create cluster through console you can referral https://cloud.google.com/sdk/gcloud/reference/dataproc/clusters/create#--properties and to use command like:
gcloud.cmd --properties 'dataproc:dataproc.logging.stackdriver.enable=false'
- Not able to create Virtual Machine
- What is the difference between Google App Engine and Google Compute Engine?
- GCP VM Auto Stop and Start setup using GCP Instance schedule
- Memory reduction Tensorflow TPU v2/v3 bfloat16
- How to use Google Cloud Shell with the new Windows Terminal
- setting up global gcp vm
- Replication Controller VS Deployment in Kubernetes
- How to SSH to docker container in kubernetes cluster?
- Is there a way to know the creator of a GCP instance with gcloud SDK?
- GCP auto shutdown and startup using Google Cloud Schedulers
- How to connect to remote PostgreSQL server on google compute engine?
- How do I check the uptime of a GCE instance using Stackdriver?
- Can I map port(s) from a container to its host on GCE using `gcloud create-with-container`?
- Cannot use gcloud compute ssh command in python subprocess
- Why does Google Cloud Storage freeze when I try to upload a large folder (2.5GB of images)?
- Is there an API to list all google compute regions
- Docker connection issues on Compute Engine
- Google Cloud VM Files Deleted after session disconnect
- Programmatically get current Service Account on GCP
- Kubernetes Ingress (GCE) keeps returning 502 error
- powershell@2 task download is failing on self hosted gcp windows 2022 agents
- By default , port 80 is open on google cloud compute engine VM. How to close it?
- GCP: Persist source-based route for nic1 on a Debian VM
- Authenticate Google Compute Engine (GCE) to Pull Image from Google Container Registry (GCR)
- what's difference between dataproc cluster on GKE vs Compute engine?
- Create compute engine from instance-template using gcloud CLI
- Automate GCP persistent disk initialization
- Can't connect to my google cloud VM instance through tcp using python
- How to automatically exit/stop the running instance
- Instance group denied permission to attach tags to its instances