Search code examples
azure-data-factoryazure-vm

Canonical way to call 'Run Command' on a VM from Datafactory?

I need to execute various batch scripts located in a VM within my Resource Group. The call of these scripts needs to be orchestrated by Datafactory.

I learned about Run Command in VM, and which can be used via PowerShell or Rest API. I was wondering if any of these ways is recommended to include in a Datafactory pipeline.

My approach at this point would be to use a Web Activity to do the API calls, one for each script.

My question: Is there a more recommended way to do this? Do you see any bottleneck or situation that may be a problem with this approach?

Solution

  • I think it is the easiest way to achieve this, one point you need to pay attention to is the Authentication when using the Web activity, remember to choose the Managed Identity, and specify the Resource with https://management.azure.com/.

    enter image description here

    To call the REST API - Virtual Machines Run Commands - Run Command successfully in Web activity, you also need to give the RBAC role to the Managed Identity(MSI).

    Navigate to the subscription or the VM in the portal -> Access control (IAM) -> Add -> Add role assignment -> search for the name of your ADFv2 and add it as an Owner/Contributor role.

    enter image description here

    If you are not familiar with Managed Identity(MSI), refer to this doc. Also make sure your ADFv2 has the MSI, when creating data factory through Azure portal or PowerShell, managed identity will always be created automatically, if not, follow this to generate managed identity.