Connecting to Azure Databricks with user assigned managed identity

Question

TL;DR: Authentication to Databricks using managed identity fails due to wrong audience claim in the token.

Technical details: When acquiring token to access databricks using managed identity (with http://169.254.169.254/metadata/identity/oauth2/token API), the returned token audience is 'spn:2ff814a6-3304-4ab8-85cb-cd0e6f879c1d' instead of '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d'.

When trying to use the acquired token, I get the following error: Error 400 io.jsonwebtoken.IncorrectClaimException: Expected aud claim to be: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d, but was: spn:2ff814a6-3304-4ab8-85cb-cd0e6f879c1d.

It seems that the audience is always prefixed with 'spn:' in case of the resource being a guid and not a url.

I also tried to use the databricks app url ('https://azuredatabricks.net/') as the resource, but the token was not accepted here also.

Is this a known issue? Are there any workarounds (other than using the service principal method)?

Thanks!

Answer 1

I can also reproduce your issue, it looks like a bug, using managed identity with Azure Container Instance is still a preview feature.

I also test the same user-assigned managed identity with a Linux VM with the same curl command, it works fine.