I am trying to run some code using databricks-connect, but I am suddenly running into this error,
22/06/24 13:24:48 ERROR SparkClientManager: Fail to get the SparkClient
java.util.concurrent.ExecutionException: com.databricks.service.SparkServiceConnectionException: Invalid shard address
Everything was working fine until today. It looks like some of my colleagues are getting the same error as well, out of nowhere.
I came across this post as well, but it only provides the link to an article that explains how to set up db-connect on VS Code,
https://community.databricks.com/s/question/0D53f00001fckBSCAY/databricksconnect-invalid-shard-address
How can this be resolved?
That's issue that did arise recently due the way how the host is parsed - to fix it, remove trailing slash (/) from the Databricks workspace host. I.e., https://adb-131442342.9.azuredatabricks.net/ should be https://adb-131442342.9.azuredatabricks.net