How to disable logs for whole cluster in gce
Could it be possible for already created (Yarn/Hadoop) cluster to disable logging for all servers inside ?
I can't find anything like it. Is there anything in Dataproc or Compute Engine which can help me to disable the logs ?
With little help and suggestion from Google support, there is complete solution to skip logging for whole yarn/hadoop cluster.
That can be possible only when create new cluster from dataproc either by google cloud page or console.
Property which should to be set in cluster properties filed:
dataproc:dataproc.logging.stackdriver.enable to be false
More info at: https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/cluster-properties
If you create cluster through console you can referral https://cloud.google.com/sdk/gcloud/reference/dataproc/clusters/create#--properties and to use command like:
gcloud.cmd --properties 'dataproc:dataproc.logging.stackdriver.enable=false'
- Im creating a VM using terraform code in GCP. I would like to inject a script that runs when the vm is created upon startup
- Can we use cron job to start the Compute engine at particular time?
- Error "Could not find 'java' executable in JAVA_HOME or PATH" while installing sonar-scanner with sonarqube google compute instance
- Google Cloud Console "loading" forever
- How to enable logs from GCP instances to Google logs exporter?
- schedule autoscaler in terraform for GCP?
- Check TPU workload/utilization
- Programmatically get current Service Account on GCP
- How to SSH to docker container in kubernetes cluster?
- Are GCP default firewall-rules a security-concern?
- Google Cloud platform performance: how many users
- Google Cloud Compute Engine refusing connections despite firewall rule
- Google Compute Engine and ADC Application Default Credentials
- Is there float32 precision on Google TPU?
- Download Directory from Google Cloud Compute Engine
- GCloud: Creating instance template with docker container and GPU
- Google GCE Container OS docker.io URL format
- How to connect to GCP VM instance with password using SSH?
- Rsync to Google Compute engine Instance from Jenkins
- Issue with gcloud compute reset-windows-password using service account in Google Cloud
- The resource 'projects/<my project>' was not found" error when trying to get list of running instances
- GCP Committed Usage Discount(CUD) Utilization REST API
- Managed Instance Group in Google Cloud stuck in failed state after deletion attempt
- How to automatically exit/stop the running instance
- What is the difference between Google App Engine and Google Compute Engine?
- GCP VM Auto Stop and Start setup using GCP Instance schedule
- Memory reduction Tensorflow TPU v2/v3 bfloat16
- How to use Google Cloud Shell with the new Windows Terminal
- setting up global gcp vm
- Replication Controller VS Deployment in Kubernetes