Search code examples
sshcontinuous-integrationgitlabgitlab-cigitlab-ci-runner

Getting GitLab CI to clone private repositories


I have GitLab & GitLab CI set up to host and test some of my private repos. For my composer modules under this system, I have Satis set up to resolve my private packages.

Obviously these private packages require an ssh key to clone them, and I have this working in the terminal - I can run composer install and get these packages, so long as I have the key added with ssh-add in the shell.

However, when running my tests in GitLab CI, if a project has any of these dependencies the tests will not complete as my GitLab instance needs authentication to get the deps (obviously), and the test fails saying Host key verification failed.

My question is how do I set this up so that when the runner runs the test it can authenticate to gitlab without a password? I have tried putting a password-less ssh-key in my runners ~/.ssh folder, however the build wont even add the key, "eval ssh-agent -s" followed by ssh-add seems to fail saying the agent isn't running...


Solution

  • I'm posting this as an answer since others weren't completely clear and/or detailed IMHO

    Starting from GitLab 8.12+, assuming the submodule repo is in the same server as the one requesting it, you can now:

    1. Set up the repo with git submodules as usual (git submodule add git@somewhere:folder/mysubmodule.git)

    2. Modify your .gitmodules file as follows

       [submodule "mysubmodule"]
         path = mysubmodule
         url = ../../group/mysubmodule.git
      

      where ../../group/mysubmodule.git is a relative path from your repository to the submodule's one.

    3. Add the following lines to gitlab-ci.yml

       variables:
         GIT_SUBMODULE_STRATEGY: recursive
      

      to instruct the runner to fetch all submodules before the build.

    Caveat: if your runner seems to ignore the GIT_SUBMODULE_STRATEGY directive, you should probably consider updating it.

    (source: https://docs.gitlab.com/ce/ci/git_submodules.html)