Search code examples
azureazure-data-factorydatabrickstagging

Apply Custom Tags to the Interactive cluster in Data bricks

I'm working on implementing custom tags to track costs associated with Databricks jobs. Successfully, I added tags to the job cluster by modifying the ADF JSON pipeline schema. However, I'm struggling to apply tags to interactive clusters. Below is the JSON schema I've been using. Could anyone guide me on how to proceed with tagging interactive clusters?

Tried below tagging approach: I'm able to view the default tags enforced by my organization, but I'm having trouble applying custom tags. While the base parameters are visible, the custom tag doesn't appear.

{
                "name": "Testing",
                "type": "DatabricksNotebook",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "notebookPath": "/test/DB_test",
                    "baseParameters": {
                        "Team": "Football"
                    }
                },
                "linkedServiceName": {
                    "referenceName": "linkedserver_databricks",
                    "type": "LinkedServiceReference"
                },
                "parameters": {
                    "Team": "Cricket"
                }
            }
Solution

  • Only when you are creating new job cluster you will get the option to add custom tags whereas using the existing interactive cluster you won't get that option.

    Because the existing in the sense it is already created in the databricks workspace and you are selecting it to only run the task. So, you can only modify or add the tags for existing cluster through workspace or rest Api.

    You follow below to add tags in workspace.

    enter image description here

    next add tags and save.

    enter image description here

    Also you add or update this using rest api.