Search code examples
azureazure-hdinsightazure-monitoringazure-monitor-workbooks

In Azure does Ambari Web UI provide information to suggest how to scale

Releated to AZure Cloud

Batch processing will run daily and must: ✑ Scale to minimize costs ✑ Be monitored for cluster performance

Dose Ambari Web UI will monitor clusters and provide information to suggest how to scale?

Is Ambari tool is recommended (or) Instead of using Ambari tool do i need to use "Azure Log Analytics" and "HDInsight cluster management solutions".

Got got confused need some clarification.

Solution

  • You can use Azure HDInsight's free Autoscale feature can automatically increase or decrease the number of worker nodes in your cluster based on previously set criteria. You set a minimum and maximum number of nodes during cluster creation, establish the scaling criteria using a day-time schedule or specific performance metrics, and the HDInsight platform does the rest.

    How it works?

    The Autoscale feature uses two types of conditions to trigger scaling events: thresholds for various cluster performance metrics (called load-based scaling) and time-based triggers (called schedule-based scaling). Load-based scaling changes the number of nodes in your cluster, within a range that you set, to ensure optimal CPU usage and minimize running cost. Schedule-based scaling changes the number of nodes in your cluster based on operations that you associate with specific dates and times.

    Auto Scale - Load-based:

    enter image description here

    Auto Scale - Scheduled-based:

    enter image description here

    Reference: Automatically scale Azure HDInsight clusters