Search code examples
azureazure-devopsazure-data-factory

How to refresh the access token for using REST API in Azure?

How do I refresh my access token while implementing a Web Activity on Azure?

I have done the following:

  1. Created a Web Activtiy to run based on some filters using: https://learn.microsoft.com/en-us/rest/api/datafactory/pipeline-runs/query-by-factory?tabs=HTTP
 {
                "name": "get pp1 runs based on filters",
                "description": "get pp1 runs based on filters.",
                "type": "WebActivity",
                "dependsOn": [
                    {
                        "activity": "Lookup1",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "url": "POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/queryPipelineRuns?api-version=2018-06-01",
                    "method": "POST",
                    "headers": {
                        "Content-Type": "application/json",
                        "Authorization": "https://login.microsoftonline.com/common/oauth2/authorize"
                    },
                    "body": {
                        "lastUpdatedAfter": "2023-07-23T12:00:36.33Z",
                        "lastUpdatedBefore": "2023-07-24T12:01:35Z",
                        "filters": [
                            {
                                "operand": "pp1",
                                "operator": "Equals",
                                "values": [
                                    "pipeline1"
                                ]
                            }
                        ]
                    },
                    "authentication": {
                        "credential": {
                            "referenceName": "credential_ref",
                            "type": "CredentialReference"
                        },
                        "type": "UserAssignedManagedIdentity"
                    }
                }
            },

I am using Authentication as User Assigned Managed Identity and Integration Runtime as AutoResolveIntegrationRuntime

When I am debugging this, it fails on this webactivtiy and gives me the error: RefreshMsiAccessToken: accessToken null

I read a few links and this is because the token expires every 30 mins to 60 mins. How do I continiously ( or in some time interval ) call the access token every one hour?

Solution

  • As you Created a Web Activity to run based on some filters using: query-by-factory and Authentication as User Assigned Managed Identity you need not to provide Authorization header in web activity.

    You can follow below steps:

    1. Create a User Assigned Managed Identity in your resource group. enter image description here
    2. Add your User Assigned Managed Identity in Data Factory managed identity. enter image description here
    3. In Data Factory You need to provide Data Factory Contributor role to your managed Identity. enter image description here
    4. Now in data factory take a web activity and pass the url with authentication as User Assigned Managed Identity and settings as below: enter image description here

    My pipeline.json:

                {
                "name": "Web1",
                "type": "WebActivity",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "url": "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/queryPipelineRuns?api-version=2018-06-01",
                    "method": "POST",
                    "body": {
                        "lastUpdatedAfter": "2023-07-24T00:00:00.3345758Z",
                        "lastUpdatedBefore": "2023-07-26T00:00:00.3686473Z",
                        "filters": [
                            {
                                "operand": "PipelineName",
                                "operator": "Equals",
                                "values": [
                                    "pipeline1"
                                ]
                            }
                        ]
                    },
                    "authentication": {
                        "resource": "https://management.azure.com/",
                        "credential": {
                            "referenceName": "credential1",
                            "type": "CredentialReference"
                        },
                        "type": "UserAssignedManagedIdentity"
                    }
                }
            }

    Output:

    Pipeline Ran succesfully: enter image description here