Search code examples
gitlabgitlab-cigit-submodules

How do I pass credentials to pull a submodule in a Gitlab CI script?

I have several projects, each in their own repository, that import a common library which has its own repository as well. So, the .gitmodules file includes the library with the full name:

Submodule 'xx/yy' (https://gitlab.com/xx/yy.git) registered for path 'xx/yy'

but this doesn't work:

Fatal: could not read Username for 'https://gitlab.com': No such device or address

the CI script is very simple:

image: mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview9-alpine3.9

variables:
  GIT_SUBMODULE_STRATEGY: recursive

stages:
    - build

before_script:
    - "cd xx"
    - "dotnet restore"

build:
    stage: build
    script:
      - "cd xx"
      - "dotnet build"

The old answer was: GitLab pull submodules inside CI

but things have changed and we can, according to the docs, have submodules that don't have a relative path, as written here: https://docs.gitlab.com/ce/ci/git_submodules.html

Solution

  • tldr; like this:

    # .gitlab-ci.yml

stages:
  - build

job1:
  stage: build
  before_script:
    - git config --global credential.helper store
    - git config --global credential.useHttpPath true
    - |
        git credential approve <<EOF
        protocol=https
        host=gitlab.com
        path=my-group/my-submodule-repo.git
        username=${CI_DEPENDENCY_PROXY_USER}
        password=${CI_DEPENDENCY_PROXY_PASSWORD}
        EOF
    - git submodule update --init --recursive

  script:
    - echo "Let's start the build..."

    Explanation

    The stages: - build and job1: stage: build declarations are boilerplate --- they inform the gitlab ci machinery that there exists one stage (named build) and one job that "belongs" to this stage.

    The before_script part details things that need to happen early in the job --- everything thereunder must complete before script is started.

    The git config --global credentials.helper tells git to use the credentials helper named "store". By default, this is a cleartext file located at ~/.git-credentials containing newline-delimited username-password-decorated URIs, each corresponding to a given git remote added by the user.

    The git config --global credentials.useHttpPath tells git not to ignore the path attribute for any call (explicit or otherwise) to git credential. This is not strictly necessary, but rather good practice when, for example, you have multiple git remotes on the same host.

    The git credential approve reads standard input (expressed as a heredoc) and passes the given credential to the credential.helper, namely store, to be written into ~/.git-credentials.

    The git submodule update --init --recursive populates the existing (but as yet incomplete) superproject worktree with the content referenced by .gitmodules.

    Assumptions

    This aforementioned example makes the following assumptions:

    References: