ValueError: cannot configure default credentials - Visual studio and Databricks

Question

I am trying to connect to Databricks with Visual Studio Code Databricks extension. Get following error:

databricks-connect test

In the extension everything looks fine I am logged into Databricks. Shouldnt I be able to connect to Databricks as User to Machine without a token or any further configurations?

Answer 1

cannot configure default credentials, please check [https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication](https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication "https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication") to configure credentials for your preferred authentication method.

Databricks is unable to locate or configure the necessary authentication credentials for your environment. That may be the reason to get above error. According to the MS document below are the authenticate methods for client unified authentication:

1. Azure Databricks personal access token authentication
2. Use a service principal to authenticate with Azure Databricks (OAuth M2M)
3. OAuth user-to-machine (U2M) authentication
4. Azure managed identities authentication
5. MS Entra service principal authentication
6. Azure CLI authentication

According to this databricks recommends OAuth U2M authentication, which is easy to configure using the Databricks extension for Visual Studio Code. So, follow below procedure to connect databricks using extension in VS code:

Go to databricks extension in VS code click on Initialize new project, it will go to command palette, provide your databricks workspace endpoint as shown below:

Create New Databricks CLI Profile as shown below:

It will go to browser, login to data bricks workspace it will show as below:

Create a new project as prompted instructions, after creation you will be able to connect databricks successfully as shown below: