Azure credentials have not been set up or have expired, please run Connect-AzAccount

Question

Related to the following question (Azure DevOps - Custom Task - PowerShell with Azure Authentification) I am using now Connect-AzAccount to login by using a Azure DevOps Custom Task.

In the next step I want to run multiple jobs in parallel to first manipulate and then deploy certain Azure resources via templates. Working with AzureRM this worked without problems but after switching to Az I always get the error

Your Azure credentials have not been set up or have expired, please run Connect-AzAccount to set up your Azure credentials.

I already updated the Az modules to the most current version.

Here is how I do it:

Authenticate

try {
    $endpoint = Get-VstsEndpoint -Name $serviceName -Require
    if (!$endpoint) {
        throw "Endpoint not found..."
    }
    $subscriptionId = $endpoint.Data.SubscriptionId
    $tenantId = $endpoint.Auth.Parameters.TenantId
    $servicePrincipalId = $endpoint.Auth.Parameters.servicePrincipalId
    $servicePrincipalKey = $endpoint.Auth.Parameters.servicePrincipalKey

    $spnKey = ConvertTo-SecureString $servicePrincipalKey -AsPlainText -Force
    $credentials = New-Object System.Management.Automation.PSCredential($servicePrincipalId,$spnKey)

    Connect-AzAccount -ServicePrincipal -TenantId $tenantId -Credential $credentials
    Select-AzSubscription -SubscriptionId $subscriptionId -Tenant $tenantId

    $ctx = Get-AzContext
    Write-Host "Connected to subscription '$($ctx.Subscription)' and tenant '$($ctx.Tenant)'..."
} catch {
    Write-Host "Authentication failed: $($_.Exception.Message)..." 
}

Deploy in parallel

foreach ($armTemplateFile in $armTemplateFiles) {
    $logic = {
        Param(
            [object] 
            [Parameter(Mandatory=$true)]
            $ctx,

            [object] 
            [Parameter(Mandatory=$true)]
            $armTemplateFile,

            [string] 
            [Parameter(Mandatory=$true)]
            $resourceGroupName
        )

        ####################################################################
        # Functions
        function Format-ValidationOutput {
            param ($ValidationOutput, [int] $Depth = 0)
            Set-StrictMode -Off
            return @($ValidationOutput | Where-Object { $_ -ne $null } | ForEach-Object { @('  ' * $Depth + ': ' + $_.Message) + @(Format-ValidationOutput @($_.Details) ($Depth + 1)) })
        }

        ####################################################################
        # Logic

        # Some template manipulation


        $paramFileContent | ConvertTo-Json -Depth 100 | Set-Content -Path $paramTemplateFile.FullName
        $templateFileContent | ConvertTo-Json -Depth 100 | Set-Content -Path $armTemplateFile.FullName

        ####################################################################
        # Test Deployment
        $ErrorMessages = Format-ValidationOutput (Test-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
                                                                                        -TemplateFile $armTemplateFile.FullName `
                                                                                        -TemplateParameterFile $paramTemplateFile.FullName `
                                                                                        -DefaultProfile $ctx)
        if ($ErrorMessages) {
            Write-Host '', 'Validation returned the following errors:', @($ErrorMessages), '', 'Template is invalid.'
        }
        else { # Deploy

            New-AzResourceGroupDeployment -Name (($armTemplateFile.Name).Split(".")[0] + ((Get-Date).ToUniversalTime()).ToString('MMddHHmm')) `
                                                -ResourceGroupName $resourceGroupName `
                                                -TemplateFile $armTemplateFile.FullName `
                                                -TemplateParameterFile $paramTemplateFile.FullName `
                                                -Force `
                                                -ErrorVariable ErrorMessages `
                                                -DefaultProfile $ctx
            if ($ErrorMessages) {
                Write-Host '', 'Template deployment returned the following errors:', @(@($ErrorMessages) | ForEach-Object { $_.Exception.Message.TrimEnd("`r`n") })
            }
        }
    }
    Start-Job $logic -ArgumentList (Get-AzContext), $armTemplateFile, $ResourceGroupName
}

While (Get-Job -State "Running")
{
    Start-Sleep 10
    Write-Host "Jobs still running..."
}

Get-Job | Receive-Job

Answer 1

For some reason (I don't know) passing the context to a background job does not work anymore since moving to the Az modules (also with the most current version (1.4.0)). I tried either using Save-AzContext with Import-AzContext or getting the context via Get-AzContext and passing it to the background job (with Set-AzContext or directly using -DefaultProfile (this approach worked with the AzureRm modules)).

A workaround that works for me now is to authenticate each background job individually.

For the job definition:

foreach ($armTemplateFile in $armTemplateFiles) {
    $logic = {
        Param(
            [object] 
            [Parameter(Mandatory=$true)]
            $endpointInput,

            [object] 
            [Parameter(Mandatory=$true)]
            $armTemplateFile,

            [string] 
            [Parameter(Mandatory=$true)]
            $resourceGroupName
        )

        ###########################
        #Login
        Write-Host "Authenticating..."
        try {
            $endpoint = $endpointInput
            if (!$endpoint) {
                throw "Endpoint not found..."
            }
            $subscriptionId = $endpoint.Data.SubscriptionId
            $tenantId = $endpoint.Auth.Parameters.TenantId
            $servicePrincipalId = $endpoint.Auth.Parameters.servicePrincipalId
            $servicePrincipalKey = $endpoint.Auth.Parameters.servicePrincipalKey

            $spnKey = ConvertTo-SecureString $servicePrincipalKey -AsPlainText -Force
            $credentials = New-Object System.Management.Automation.PSCredential($servicePrincipalId,$spnKey)

            Connect-AzAccount -ServicePrincipal -TenantId $tenantId -Credential $credentials
            Select-AzSubscription -SubscriptionId $subscriptionId -Tenant $tenantId

            $ctx = Get-AzContext
            Write-Host "Connected to subscription '$($ctx.Subscription)' and tenant '$($ctx.Tenant)'..."
        } catch {
            Write-Host "Authentication failed: $($_.Exception.Message)..." 
        }

For starting the job:

Start-Job $logic -Name $jobName -ArgumentList $endpoint, $armTemplateFile, $ResourceGroupName