Azure Python Function app fails to import packages when deployed via Azure DevOps

Question

Scope: I have a python function app that runs as expected locally and when deployed via VS Code.

The problem arises when deploying the function app to a prod resource group (and function app).

Differences between two deployments:

• dev function app (serverless), not Vnet integrated, deployed via right-click>deploy to function app
• prod function app (premium plan), Vnet integrated, deployed via Azure DevOps pipeline, that looks like so:

trigger:
- none

variables:
  # Azure Resource Manager connection created during pipeline creation
  azureServiceConnectionId: '******'

  # Web app name
  webAppName: '*******'

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

  # Environment name
  environmentName: 'PROD'

  # Project root folder. Point to the folder containing manage.py file.
  projectRoot: $(System.DefaultWorkingDirectory)

  # Resource group
  resourceGroupName : '********'

  # Python version: 3.11
  pythonVersion: '3.11'

stages:
- stage: Build
  displayName: Build stage
  jobs:
  - job: BuildJob
    pool: Default
    steps:
    - script: |
        pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
      workingDirectory: $(projectRoot)
      displayName: "Install requirements"

    - task: ArchiveFiles@2
      displayName: 'Archive files'
      inputs:
        rootFolderOrFile: '$(projectRoot)'
        includeRootFolder: false
        archiveType: zip
        archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
        replaceExistingArchive: true

    - upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      displayName: 'Upload package'
      artifact: drop

- stage: Deploy
  displayName: 'Deploy Web App'
  dependsOn: Build
  condition: succeeded()
  jobs:
  - deployment: DeploymentJob
    pool: Default
    environment: $(environmentName)
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureFunctionApp@2
            displayName: 'Deploy Azure Web App'
            inputs:
              connectedServiceNameARM: '$(azureServiceConnectionId)'
              appType: 'functionAppLinux'
              appName: '$(webAppName)'
              package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
              runtimeStack: 'PYTHON|3.11'
              appSettings: '-KeyvaultURL ***** -StorageaccountURL *****'
              deploymentMethod: 'auto'

Additionally, I have tried to deploy the function with:

- task: AzureCLI@2
            displayName: 'Deploy with Azure CLI'
            inputs:
              azureSubscription: '$(azureServiceConnectionId)'
              scriptType: 'pscore'
              scriptLocation: 'inlineScript'
              inlineScript: |
                $artifactPath = "$(Pipeline.Workspace)/drop/$(Build.BuildId).zip"
                az functionapp deployment source config-zip -g $(resourceGroupName) -n $(webAppName) --src $artifactPath --verbose

The outcome was the same.

Now, the initial problem I was facing was that the functions just didn't show up, deployment is pretty much always successfull. After going down the rabbit hole, I managed to find logs that show the "Http triggers sync failed", along with modules failing to import (no specific modules - all of them).

Withing the log, I can see that the correct folder is being targeted:

/home/site/wwwroot/.python_packages/lib/site-packages/pandas/

And I can verify that those packages are indeed there:

For whatever reason, the function_app.py doesn't see the installed modules, which causes a chain reaction that leads to functions not showing up even after a successful deployment.

What I've tried thus far:

• Multiple combinations of using ENABLE_ORYX_BUILD and SCM_DO_BUILD_DURING_DEPLOYMENT in the pipeline
• Deployment type as either zipDeploy and/or package

There were some resources on SO, but nothing seems to solve this, it's fairly weird as I feel like this type of deployment must be done pretty much every day by many developers, so either I am doing something really wrong, or there is an underlying issue I don't seem to grasp.

Answer 1

After an extensive debugging session with Microsoft support we managed to find a solution.

It turns out that VS Code has a bunch of different checks when deploying a function app, whereas these steps needs to implemented manually in an Azure DevOps pipeline.

What finally did the trick was initially setting:

ENABLE_ORYX_BUILD = true
SCM_DO_BUILD_DURING_DEPLOYMENT = true
WEBSITE_RUN_FROM_PACKAGE = 0

As well as removing the install requirements part of the DevOps pipeline and changing the deploymentMethod to zipDeploy, instead of auto.

Logs for the function app can be found in the deployment center of the function app, where we managed to see the function app doing the build "within itself", and installing the needed requirements.

After the first deployment with the above settings the functions finally appeared on the function app, after which we could edit the pipeline to do the build (brought back the install requirements part), and set:

ENABLE_ORYX_BUILD = false
SCM_DO_BUILD_DURING_DEPLOYMENT = false

After the initial build was done on the app itself, the deployment with the Azure DevOps build can now be used normally.

It is still unclear if we will need to do the same next time we add an additional package to the project, if so, I will make sure to provide an update on this post.

I hope this helps someone in the future as I feel it is a fairly common use case.