GitLab CI login to private repo without DinD

Question

I'm new to GitLab and not sure if this is possible, I have on premise GitLab set up and working as well as Artifactory private repo.

I always used DinD configuration, using Docker as the main image with DinD service, then in the stages logging in and pulling different images from the private repo.

But I heard it's possible to do this without DinD to have shorter execution times. And the needed image is pulled in the beginning of a stage.

Instead of this:

image: docker:latest

services:
  - docker:18.09.8-dind

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_HOST: tcp://localhost:2375 

stages:
  - run_script

run_script:
  stage:
    run_script
  script:
    - docker login -u $DOCKER_TECHNICAL_USER -p $DOCKER_TECHNICAL_USER_PASSWORD $DOCKER_REGISTRY_URL
    - docker pull artifactory.example.com:5000/repo/powershell-core:6.3
    - docker run -it artifactory.example.com:5000/repo/powershell-core:6.3 pwsh -command "get-process"

I want to do it like this(whole yml):

stages:
  - run_script

run_script:
  before_script:
    - docker login -u $DOCKER_TECHNICAL_USER -p $DOCKER_TECHNICAL_USER_PASSWORD $DOCKER_REGISTRY_URL
  image: artifactory.example.com:5000/repo/powershell-core:6.3
  stage:
    run_script
  script:
    - pwsh -command "get-process"

If I try this it says it can't authenticate:

ERROR: Job failed: image pull failed: rpc error: code = Unknown desc = Get https://artifactory.example.com:5000/v2/repo/powershell-core/manifests/6.3: unknown: Authentication is required

Is this impossible or do I have a mistake somewhere and it's fixable?

Answer 1

Your problem

Your CI is failing because it doesn't know the credentials for artifactory.example.com when downloading the base image artifactory.example.com:5000/repo/powershell-core:6.3. For you to understand, I'll explain the different steps of the 2 CIs you gave and then I'll give tracks for a solution.

First CI (DinD)

In your first CI (the one using DinD), what happens is :

• The Gitlab-runner executor download the image docker:18.09.8-dind and then it starts the image as a service for your CI
• The Gitlab-runner executor download the base image docker:latest and then it uses it to execute your job run_script
• In your docker docker:latest, the job run_script log in your private repository with your credentials and through the DinD service
• Then it download your image artifactory.example.com:5000/repo/powershell-core:6.3 and run a script using it, all through the DinD service

Second CI

In this one, you are trying just to execute your image artifactory.example.com:5000/repo/powershell-core:6.3 and run a script using it. You are right, for a simple goal like that no DinD is necessary. Here is the analysis of what your CI is doing :

• The Gitlab-runner executor try to download the base image specified by the job run_script : artifactory.example.com:5000/repo/powershell-core:6.3
• The repository artifactory.example.com ask for credentials
• The executor doesn't know any credentials so it returns an error and it stops

As you can see, in this case, the job run_script was never executed because the executor failed to download the base image specified by the job. The before_script part of the job, which is responsible for the login, is not executed either because the before_script is executed in the base image and this last one couldn't be downloaded by the executor.

Thus, the solution is simply to give the credentials to the executor so it can login and then download the base image of your job. Also, the before_script part of your job should be removed because it is not executed at the time you intended and therefore not necessary.

Solution

So what you seek is a way to give the credentials for your repository artifactory.example.com to the Gilab-runner executor your job is using. Sadly, there is no unique way to do such a thing because it depends on the executor that you are using. Since you didn't specify the executor in your question, I'll give the solutions that, I think, are the most used and convenient for Docker and Kubernetes.

Using DOCKER_AUTH_CONFIG

One solution which works with several executors is to define a (secret) variable directly in Gitlab as explained in this Gitlab documentation.

Here, I present the second method to prepare your credentials. Adapt the following to generate your own auth fied :

echo -n "USERNAME:PASSWORD" | base64
VVNFUk5BTUU6UEFTU1dPUkQ=

Then create a variable DOCKER_AUTH_CONFIG in Gitlab with the following content that you adapted first :

{
  "auths": {
    "artifactory.example.com:5000": {
      "auth": "VVNFUk5BTUU6UEFTU1dPUkQ="
    }
  }
}

Using this method, your CI simply becomes :

stages:
  - run_script

run_script:
  image: artifactory.example.com:5000/repo/powershell-core:6.3
  stage:
    run_script
  script:
    - pwsh -command "get-process"

This solution works for most cases. However, depending on your executor and the access you have to it, you may want to use other specific solutions.

Kubernetes executor

If you are using Kubernetes executor, the simplest solution is to create a secret containing the credentials for your repository. Using the runner's Helm chart, this secret can be given during the chart installation using the runners.imagePullSecrets key which is described in the values.yaml of the chart. This solution uses the Kubernetes' mechanism to authenticate into the registry as explained in the Kubernetes documentation.

I give this example to illustrate the fact that the runner's authentication into the registry is using mechanisms that are independent of the runner itself. In this case, it is Kubernetes' mechanisms. Therefore, there is no unique way for the runner to authenticate.