This is my pipeline output from the "Checkout repo@main to s" task:
......
git submodule sync
Synchronizing submodule url for 'submodule01'
Synchronizing submodule url for 'submodule02'
Synchronizing submodule url for 'submodule03'
Synchronizing submodule url for 'submodule04'
Synchronizing submodule url for 'submodule05'
git --config-env=http.https://myorg@dev.azure.com.extraheader=env_var_http.https://myorg@dev.azure.com.extraheader submodule update --init --force --depth=1
fatal: Cannot prompt because terminal prompts have been disabled.
fatal: Cannot prompt because terminal prompts have been disabled.
fatal: could not read Username for 'https://myorg.visualstudio.com': terminal prompts disabled
Unable to fetch in submodule path 'submodue01'; trying to directly fetch 834654e979acb0c929794db7261e6f8a73b6e37e:
fatal: Cannot prompt because terminal prompts have been disabled.
fatal: Cannot prompt because terminal prompts have been disabled.
fatal: could not read Username for 'https://myorg.visualstudio.com': terminal prompts disabled
fatal: Fetched in submodule path 'submodue01', but it did not contain 834654e979acb0c929794db7261e6f8a73b6e37e. Direct fetching of that commit failed.
##[error]Git submodule update failed with exit code: 128
All of the repos are in the same project.
I understand that the git config-env=... line is supposed to be pulling the auth token from the environment and setting it in the git config for use in authenticating against the submodules. If the value were empty, it would throw the exception here, but it does not. Still, the authentication fails as it defaults to an attempt at prompting for credentials.
I have already done the following for each submodule repo:
Project Setting > Repositories > Security > [repo] Build Service > (verified Allow is inherited from Project Collection Build Service Accounts)
Project Setting > Repositories > Security > Project Collection Build Service Accounts (Allow: Contribute, Create branch, Create tag, and Read)
Project Setting > Repositories > Pipeline permissions + (added my pipeline)
Pipeline Settings:
Project Setting > Pipelines > Settings > Protect access to repositories in YAML pipelines (Off)
In my pipeline I have the following:
steps:
- checkout: self
submodules: true
persistCredentials: true
I even added an explicit reference to the repo resource:
resources:
repositories:
- repository: name
type: git
name: project/_git/repo
My .gitmodules file looks like this:
[submodule "submodue01"]
path = submodue01
url = https://myorg.visualstudio.com/repos/_git/submodue01
[submodule "submodue02"]
path = submodue02
url = https://myorg.visualstudio.com/repos/_git/submodue02
[submodule "submodue03"]
path = submodue03
url = https://myorg.visualstudio.com/repos/_git/submodue03
[submodule "submodue04"]
path = submodue04
url = https://myorg.visualstudio.com/repos/_git/submodue04
[submodule "submodue05"]
path = submodue05
url = https://myorg.visualstudio.com/repos/_git/submodue05
I tried creating a new pipeline (same pipeline definition, same agent, new workspace), but got the same error.
Also, this is a new behavior. It was working fine before. Furthermore, the problem seems to be isolated to the remote connection, because, if I fetch the required commit manually ahead of time (on the command line, in the agent workspace, using my own creds) then the pipeline apparently doesn't need to make that remote connection and will successfully checkout the target commit, since it's already local.
Other threads I've read mention things like ssh and manually adding a PAT, but obviously these approaches should not be necessary, since Micro$oft is clearly trying to make this work as it is.
What am I missing?
It appears that Azure has broken its backwards compatibility with the old [org].visualstudio.com style of URL. As bryanbcook said, the credentials being configured by the agent use the new dev.azure.com/[org] style of the URL. This used to work anyway, but now it doesn't. I dealt with this by reconstructing the repo to use the new style of URL with the following steps:
NOTE: I did NOT have to flip the switch to "Use the new URL" in org settings. Doing so before you have updated all your endpoints may cause problems.
First replace the remote URL for the main repo:
git remote remove origin
git remote add origin https://dev.azure.com/[org]/[project]/_git/[repo]
git pull origin main
git push -u origin main
Then, remove the submodule:
- Delete the relevant section from the .gitmodules file
- Delete the relevant section from .git/config
git update-index --remove .git\modules\[submodule]
- Delete the submodule directory from .git\modules
git add .
git commit -m "removed submodule"
Finally, add the submodule back in, using the new URL:
git submodule add https://dev.azure.com/[org]/[project]/_git/[submodule]
The pipeline was then able to successfully fetch the submodules.
Probably time to start migrating to the new URL anyway. Especially if support for the backward compatibility continues to degrade.