In google composer environment there is tab - Monitoring which has Environment overview section and very first metrics is Environment health.
I have read this in google documentation:
Environment health A timeline showing the health of the Composer deployment. Green status doesn't mean that all Airflow components were operational and DAGs were able to be run--it only reflects the status of the Composer deployment.
But not sure what's wrong with the environment and how to fix it when it happens. Even though my Environment health shows unhealthy all other metrics are healthy and even the airflow dags are running. So can you please explain me whats does this unhealthy status on Environment health means and what is the impact on airflow data pipeline and how to fix it.
The environment health timeline indicates whether or not your environment was able to run a very simple prober DAG (airflow_monitoring
) that is preinstalled in all Cloud Composer environments. If the DAG ran successfully and its status was read properly by the metrics collector, then the environment's health will show green, else it will show red.
In general, the timeline is a good indicator of if your environment is able to run task instances. However, if it shows red and you do not see any other problems with your environment, it is possible that there is a misconfiguration or bigger problem in your environment.
Some recommendations for troubleshooting this is ensuring that the airflow_monitoring
DAG is not paused, and checking the airflow-monitoring
log category in Cloud Logging. If airflow_monitoring
DAG runs are not completing successfully, then it is recommended that you troubleshoot it as a normal Airflow issue to unstick it before looking into Composer-specific components.