Search code examples
azure-databricks

DBU Range for cluster: how is DBU consumption determined

The cluster summary in databricks looks like the following:

enter image description here

It shows a DBU range of 1.5-2.25 DBU/h. In which scenario are 1.5 DBU/h used, in which scenario are 2.25 DBU/h used? I guess it's because of the workers (but 1.5x2=3)

Solution

  • Converting the discussion in the comments so it might help other community members:

    If you deselect auto scaling, then fixed number of workers will be assigned to the cluster which have their respective DBU. But when you are using auto-scaling option, then the dbu depends on the number of workers utilized. In your case, it is between 1.5 to 2.25. So, this will be determined by the operations that you are going to perform.

    • The DBU/h is allocated based on the number of workers. Each worker is allocated a DBU/h of 0.75 DBU/h and hence it is 1.5 DBU/h for 1 worker and 2.25 DBU/h for 2 workers and so on.

    • The following is a reference image indicating the same:

    enter image description here