Search code examples
dockercloud-foundrydocker-registryconcourse

How do you provide access credentials for private docker registry in concourse?

Until recently I was able to deploy docker images from Concourse to cloud foundry as shown in this concourse pipeline snippet:

resources:
- name: cf-build-in
  type: cf
  source:
    api: ((cf-api-endpoint))
    username: ((cf-username-email))
    password: ((cf-password))
    organization: ((cf-organization))
    space: Development
    skip_cert_check: false

jobs:
- name: deploy-build-in-cf-private
  plan:
  - get: git
    passed: [build-private]
    trigger: false
  - put: cf-build-in
    params:
      manifest: git/manifest-private.yml
      docker_username: ((docker-registry-username))
      docker_password: ((docker-registry-password))
      environment_variables:
        CF_DOCKER_PASSWORD: ((docker-registry-password))

I have recently upgraded to Concourse 3.9.2 (latest as of writing), but am now experiencing issues, getting this error when running the pipeline above:

Incorrect Usage: '--docker-image, -o' and '--docker-username' must be used together.
FAILED

NAME:
   push - Push a new app or sync changes to an existing app

USAGE:
   cf push APP_NAME [-b BUILDPACK_NAME] [-c COMMAND] [-f MANIFEST_PATH | --no-manifest] [--no-start]
   [-i NUM_INSTANCES] [-k DISK] [-m MEMORY] [-p PATH] [-s STACK] [-t HEALTH_TIMEOUT] [-u (process | port | http)]
   [--no-route | --random-route | --hostname HOST | --no-hostname] [-d DOMAIN] [--route-path ROUTE_PATH]

   cf push APP_NAME --docker-image [REGISTRY_HOST:PORT/]IMAGE[:TAG] [--docker-username USERNAME]
   [-c COMMAND] [-f MANIFEST_PATH | --no-manifest] [--no-start]
   [-i NUM_INSTANCES] [-k DISK] [-m MEMORY] [-t HEALTH_TIMEOUT] [-u (process | port | http)]
   [--no-route | --random-route | --hostname HOST | --no-hostname] [-d DOMAIN] [--route-path ROUTE_PATH]

   cf push -f MANIFEST_WITH_MULTIPLE_APPS_PATH [APP_NAME] [--no-start]

ALIAS:
   p

OPTIONS:
   -b                           Custom buildpack by name (e.g. my-buildpack) or Git URL (e.g. 'https://github.com/cloudfoundry/java-buildpack.git') or Git URL with a branch or tag (e.g. 'https://github.com/cloudfoundry/java-buildpack.git#v3.3.0' for 'v3.3.0' tag). To use built-in buildpacks only, specify 'default' or 'null'
   -c                           Startup command, set to null to reset to default start command
   -d                           Domain (e.g. example.com)
   --docker-image, -o           Docker-image to be used (e.g. user/docker-image-name)
   --docker-username            Repository username; used with password from environment variable CF_DOCKER_PASSWORD
   -f                           Path to manifest
   --health-check-type, -u      Application health check type (Default: 'port', 'none' accepted for 'process', 'http' implies endpoint '/')
   --hostname, -n               Hostname (e.g. my-subdomain)
   -i                           Number of instances
   -k                           Disk limit (e.g. 256M, 1024M, 1G)
   -m                           Memory limit (e.g. 256M, 1024M, 1G)
   --no-hostname                Map the root domain to this app
   --no-manifest                Ignore manifest file
   --no-route                   Do not map a route to this app and remove routes from previous pushes of this app
   --no-start                   Do not start an app after pushing
   -p                           Path to app directory or to a zip file of the contents of the app directory
   --random-route               Create a random route for this app
   --route-path                 Path for the route
   -s                           Stack to use (a stack is a pre-built file system, including an operating system, that can run apps)
   -t                           Time (in seconds) allowed to elapse between starting up an app and the first healthy response from the app

ENVIRONMENT:
   CF_STAGING_TIMEOUT=15        Max wait time for buildpack staging, in minutes
   CF_STARTUP_TIMEOUT=5         Max wait time for app instance startup, in minutes
   CF_DOCKER_PASSWORD=          Password used for private docker repository

SEE ALSO:
   apps, create-app-manifest, logs, ssh, start
error running command: exit status 1

  1. I can't seem to find a good way to enable more debug output for the cf-resource

  2. I have tested switching to this alternative cf-cli-resource, but am receiving a similar error + they don't actually have the explicit docker_username, docker_password arguments.

  3. I tried reverting to version 3.8.0 of concourse, but am actually getting some database error, so rolling back would mean dataloss apparently.

Is there anyone running Concourse 3.9.2 deploying to CloudFoundry from a private docker registry, who could test that this should be working?

Solution

  • So the trick here was NOT to specify the docker_username: ((docker-registry-username)). Also CF_DOCKER_PASSWORD: ((docker-registry-password)) was not required in this case.

    A valid setup may then look like this:

    resources:
- name: cf-build-in
  type: cf
  source:
    api: ((cf-api-endpoint))
    username: ((cf-username-email))
    password: ((cf-password))
    organization: ((cf-organization))
    space: Development
    skip_cert_check: false

- name: repo-docker-registry
  type: docker-image
  source:
    repository: my.registry.com/repo
    username: ((docker-registry-username))
    password: ((docker-registry-password))

jobs:
- name: build-repo-docker-image
  serial_groups: [build-base]
  plan:
  - get: repo-git
    trigger: true
  - put: repo-docker-registry
    params:
      build: repo-git
      dockerfile: repo-git/Dockerfile

- name: deploy-build-in-cf-private
  plan:
  - get: git
    passed: [build-repo-docker-image]
    trigger: false
  - put: cf-build-in
    params:
      manifest: repo-git/manifest-private.yml
      docker_password: ((docker-registry-password))

    The manifest-private.yml should then specify the docker image and username (but NOT the password):

    applications:
- name: app-name
  docker:
    image: my.registry.com/repo:latest
    username: my_user