Azure Pipeline fails to checkout a submodule

Question

I have an Azure DevOps pipeline that should run for every push on the master branch. My repository has a submodule defined:

[submodule "submodules/shared_components"]
    path = submodules/shared_components
    url = ../shared_components

The submodule is basically checked out (when using git clone <link> --recurse-submodules -j8) and everything works fine. However, my Azure pipeline is unable to do a checkout on the submodule, claiming:

git submodule sync --recursive
git --config-env=http.https://[email protected]=env_var_http.https://[email protected] submodule update --init --force --depth=1 --recursive
Submodule 'submodules/shared_backend' (https://[email protected]/myorg/MyProject/_git/shared_backend) registered for path 'submodules/shared_backend'
Cloning into '/home/vsts/work/1/s/submodules/shared_backend'...
remote: TF401019: The Git repository with name or identifier shared_backend does not exist or you do not have permissions for the operation you are attempting.
fatal: repository 'https://dev.azure.com/myorg/MyProject/_git/shared_backend/' not found
fatal: clone of 'https://[email protected]/myorg/MyProject/_git/shared_backend' into submodule path '/home/vsts/work/1/s/submodules/shared_backend' failed
Failed to clone 'submodules/shared_backend'. Retry scheduled
Cloning into '/home/vsts/work/1/s/submodules/shared_backend'...
remote: TF401019: The Git repository with name or identifier shared_backend does not exist or you do not have permissions for the operation you are attempting.
fatal: repository 'https://dev.azure.com/myorg/MyProject/_git/shared_backend/' not found
fatal: clone of 'https://[email protected]/myorg/MyProject/_git/shared_backend' into submodule path '/home/vsts/work/1/s/submodules/shared_backend' failed
Failed to clone 'submodules/shared_backend' a second time, aborting
##[error]Git submodule update failed with exit code: 1

Considering the fact that the main repository (the one with the azure-pipeline.yml) gets checked out just fine, it makes no sense to me that the submodule could not be checked out, especially since both repositories are in the same Azure DevOps organization and also in the same project. This is my YAML file:

trigger:
- master

pool:
  vmImage: ubuntu-latest

variables:
  buildConfiguration: 'Release'

steps:
- checkout: self
  displayName: 'Checkout'
  submodules: recursive
  persistCredentials: true

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'Dotnet Build $(buildConfiguration)'

I also tried a different approach by creating a PAT with access to repos read and then using that, but to no avail.

- powershell: |
    $header = "AUTHORIZATION: bearer $(System.AccessToken)"
    git -c http.extraheader="$header" submodule sync
    git -c http.extraheader="$header" submodule update --init --force --depth=1

Am I missing something here?

EDIT: Forgot to post, I've already given the Build Service permissions to access the submodule repository:

Answer 1

After dealing with this for about a day, I managed to find an article named "Dealing with error TF401019 when using submodules in Azure Pipelines" written by Tim Schaeps.

Long story short, cloning submodules from pipelines seems to be disabled for security reasons. I had to disable 3 options for this work (had to disable them on organization level first before I could set a specific value for them in my Azure DevOps project):

• Limit job authorization scope to current project for non-release pipelines
• Limit job authorization scope to current project for release pipelines
• Protect access to repositories in YAML pipelines

Huge shoutout to Tim Schaeps for the article! You rock!