How to trigger an AzureML Pipeline from Azure DevOps?

Question

If we have an AzureML Pipeline published, how can we trigger it from Azure DevOps without using Python Script Step or Azure CLI Step?

The AzureML Steps supported natively in Azure DevOps include Model_Deployment and Model_Profiling.

Is there any step in Azure DevOps which can be used to directly trigger a published Azure Machine Learning Pipeline while maintaining capabilities like using Service Connections and passing environmental variables, Gated Release (Deployment)?

Edit: This process can then be used to run as an agentless job.

Answer 1

Assumptions:

1. An AzureML Pipeline is published and the REST endpoint is ready- To be referred to in this answer as <AML_PIPELINE_REST_URI>. And Published Pipeline ID is also ready- To be referred to in this answer as <AML_PIPELINE_ID>
2. You have the Azure Machine Learning Extension installed: Azure Machine Learning Extension

To Invoke the Azure Machine Learning Pipeline we use the Invoke ML Pipeline step available in Azure DevOps. It is available when running an Agentless Job.

To trigger it the workflow is as follows:

1. Create a New Pipeline. Using the Classic Editor, delete the default Agent Job 1 stage.

2. Add an agentless job:

3. Add a task to this Agentless Job:

4. Use AzureML Published Pipeline Task:

5. Use the Service Connection Mapped to the AML Workspace. You can find more on this at the official documentation

6. Choose the Pipeline to trigger using the <AML_PIPELINE_ID>:

7. Give The experiment name and Pipeline Parameters if any:

8. That's it, you can Save and Queue:

Alternatively, you can simply use the following jobs:

- job: Job_2
  displayName: Agentless job
  pool: server
  steps:
  - task: MLPublishedPipelineRestAPITask@0
    displayName: Invoke ML pipeline
    inputs:
      connectedServiceName: <REDACTED-AML-WS-Level-Service_Connection-ID>
      PipelineId: <AML_PIPELINE_ID>
      ExperimentName: experimentname
      PipelineParameters: ''