I have set up Log Analytics to take Databricks Logs.
How can I configure Databricks to access my Log Aanalytics endpoint through Databricks REST API? I want to be able to access those logs in Databricks.
For the parameters explain me which parameter from where to take.
First obtain the following information from your Azure Log Analytics workspace- Workspace ID: Can be found in the Azure Portal in the Log Analytics workspace. Shared Key: Navigate to the Log Analytics workspace in the Azure Portal, go to "Advanced Settings," and obtain the "Primary Key."
Then In the Databricks workspace, go to the cluster configuration. Add the following Spark configuration settings: spark.databricks.workspaceUrl: The URL of the Databricks workspace. spark.databricks.clusterUsageTags.cluster_name: A unique identifier for the cluster. spark.databricks.clusterUsageTags.cluster_id: The cluster ID. after its restart the cluster.
follow these docs for info. https://learn.microsoft.com/en-us/search/?terms=azure%20databricks
https://learn.microsoft.com/en-us/search/?terms=Azure%20Log%20Analytics