Search code examples
azure-devopsdatabricks

Azure DevOps Release pipeline failure with "File not found: D:\a\_tasks\*\0.1.0\index.js"

I am trying to deploy a Release Pipeline which has a "Use Python 3.9" task and a "Configure Databricks CLI" task. The "Configure Databricks CLI" task causes my pipeline to fail with the following message: File not found: D:/a/_tasks/cicd_configuredatabricks_c*18a123/0.1.0/index.js The Agent Specification is ubuntu-22.04. I imported the js file for the Release Pipeline. How can I make this pipeline work? Thank you.

I installed Azure DevOps for Databricks extension and added it manually the "Configure Databricks CLI" task. This task has two Agent Specifications: Hosted Ubuntu 1604 and Hosted VS2017.

Solution

  • I can reproduce the same error when using any of the Microsoft-hosted agents.

    The README of the DevOps for Databricks extension has clearly stated that only "Hosted Ubuntu 1604" and "Hosted VS2017" are the supported agents by Configure Databricks CLI task. The agents "Hosted Ubuntu 1604" (ubuntu-16.04) and "Hosted VS2017" (vs2017-win2016) had been deprecated and retired from Microsoft-hosted agents for more than 2 years.

    The publisher of this extension seems has not re-developed these tasks to get a good compatibility with the new Microsoft-hosted agents.

    So, the problem is with the compatibility of the extension itself.

    Instead of using the DevOps for Databricks extension, you can consider the following ways.

    1. If you still want to use the Microsoft-hosted agents to run the pipeline, you can:

      • Reference the documentation "Install or update the Databricks CLI" to use a script task (such as Bash, PowerShell, etc.) to run the command lines to install the Databricks CLI in the pipeline.
      • In the subsequent steps within the same pipeline job, you also can use the script tasks to call the Databricks CLI commands to access your Databricks workspaces.

    2. If you do not use the Microsoft-hosted agents, you can:

      • Reference above documentation to install the Databricks CLI on the local machines or VMs owned by yourself.
      • Install Self-hosted agents on your machines to run the pipelines. You also can use the script tasks to call the Databricks CLI commands to access your Databricks workspaces.