Search code examples
dockergitlab-cidockerhub

Gitlab CI service and docker hub auth

Due to new restrictions on docker hub non-authenticated pulls, how do you authenticate your docker hub account for gitlab-ci services?

Here is the sample CI config from the gitlab documentation:

# from official documentation 
services:
  - postgres:12.2 # <---- this will fail at some point because it's a non-authenticated pull

variables:
  POSTGRES_DB: nice_marmot
  POSTGRES_USER: runner
  POSTGRES_PASSWORD: ""
  POSTGRES_HOST_AUTH_METHOD: trust

This causes the followin error after a while:

ERROR: Preparation failed: Error response from daemon: 
toomanyrequests: You have reached your pull rate limit. 
You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit (executor_docker.go:188:1s)

As the services are pulled before script run, we cannot docker login in the script section. I could not find any documentation from gitlab regarding url auth or environment variable auth.

An ideal solution would not require to have admin access to gitlab-ci servers or gitlab-ci runners and would not require to setup a custom runner with pull_policy = never (which we ended up doing but it slowed down our CI drastically with a single runner bottleneck for e2e tests)

Solution

  • Check also, with GitLab 13.7 (December 2020) if the improved Dependency Proxy can help:

    Avoid Docker rate limits and speed up your pipelines

    For faster and more reliable builds, you can use the Dependency Proxy to cache the container images hosted on Docker Hub.

    But, when Docker started to enforce rate limits on pull requests from Docker Hub, you noticed that even when your image was pulled from the cache, Docker counted it against your limit.
    That’s because the Dependency Proxy was only caching the image’s layers (or blobs) and not the manifest, which contains information about how to build a given image.
    Since the manifest is required, a pull request was still required. This also means that if Docker Hub was unavailable, you couldn’t pull your image.

    Moving forward, the Dependency Proxy will cache both the image’s layers and manifest.

    So, the first time you pull alpine:latest, the image will be added to the Dependency Proxy cache and count as one pull against your rate limit.
    The next time you pull alpine:latest, it will be pulled from the cache, even if Docker Hub is unavailable and will not count against your rate limit.

    Don’t forget, as of milestone 13.6, the Dependency Proxy is available in Core. So, give it a try and let us know what you think. Or better yet, consider contributing to one of the open issues.

    See Documentation and Issue.

    And:

    Still with GitLab 13.7 (December 2020)

    Use pre-defined variables with the Dependency Proxy

    By proxying and caching container images from Docker Hub, the Dependency Proxy helps you to improve the performance of your pipelines.

    Even though the proxy is intended to be heavily used with CI/CD, to use the feature, you had to define your own variables or hard-code values in your gitlab.ci-yml file.
    This made it difficult to get started for individuals, and prevented it from being a scalable solution, especially for organizations with many different groups and projects.

    Moving forward, you can use pre-defined environment variables as an intuitive way to use the Dependency Proxy. The following variables are supported:

    • CI_DEPENDENCY_PROXY_USER: a CI user for logging in to the Dependency Proxy.
    • CI_DEPENDENCY_PROXY_PASSWORD: a CI password for logging in to the Dependency Proxy.
    • CI_DEPENDENCY_PROXY_SERVER: the server for logging in to the Dependency Proxy.
    • CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: the image prefix for pulling images through the Dependency Proxy.

    Give it a try and let us know what you think!

    See Documentation and Issue.

    This works even for private projects (December 2020)

    Use the Dependency Proxy with private projects

    You can use the GitLab Dependency Proxy to proxy and cache container images from Docker Hub. Until recently the feature was only available for public groups, preventing many of you from being able to use it.

    You can now use the Dependency Proxy with private projects.
    You can reduce your reliance on Docker Hub by caching your container images for future use.

    Because the Dependency Proxy is storing Docker images in a space associated with your group, you must authenticate with your GitLab username and password, or with your personal access token with the scope set to at least read_registry.

    See Documentation and Issue.

    With GitLab 13.9 (February 2021):

    Automatically authenticate when using the Dependency Proxy

    By proxying and caching container images from Docker Hub, the Dependency Proxy helps you to improve the performance of your pipelines.

    Even though the proxy is intended to be heavily used with CI/CD, to use the feature, you had to add your credentials to the DOCKER_AUTH_CONFIG CI/CD variable or manually run docker login in your pipeline. These solutions worked fine, but when you consider how many .gitlab-ci.yml files that you need to update, it would be better if the GitLab Runner could automatically authenticate for you.

    Since the Runner is already able to automatically authenticate with the integrated GitLab Container Registry, we were able to leverage that functionality to help you automatically authenticate with the Dependency Proxy.

    Now it’s easier to use the Dependency Proxy to proxy and cache your container images from Docker Hub and start having faster, more reliable builds.

    See Documentation and Issue.

    See GitLab 13.10 (March 2021)

    Use the Dependency Proxy with 'containerd' and Docker 20+

    The GitLab Dependency Proxy is a local proxy you can use for your frequently-accessed upstream images from Docker Hub. In the case of CI/CD, the Dependency Proxy receives a request and returns the upstream image from a registry, acting as a pull-through cache. This helps to reduce your CI minutes and increase reliability.

    However, you haven’t been able to pull images by digest, which as an immutable identifier ensures you are using the exact version of a specific image and tag. Since both containerd and Docker 20+ depend on pull-by-digest, this meant that many of you were blocked from using the Dependency Proxy.

    We are happy to say that you can now pull your container images from Docker Hub by digest. You can use the Dependency Proxy by adding the URL to your .gitlab-ci.yml file, manually pulling the image from the command line, or using a Dockerfile. Check out the documentation and start saving time on your builds.

    See Documentation and Issue.