Search code examples
continuous-integrationyamlpipelinemvconcourse

Pull from multiple SCM then mv file in Concourse CI to workdir


I've been banging my head on this one for quite some time and I cannot figure it out (I know it must be a simple thing to do though).

Currently, what I'm trying to do is pulling from two repositories (which naturally creates two separate directories) then I'm trying to move files from one directory to the other to successfully execute the Dockerfile.

Here's how my pipeline.yml file looks like:

---
jobs:
- name: build-nexus-docker-image
  public: false
  plan:
  - get: git-nexus-docker-images
    trigger: true
  - get: git-nexus-license
    trigger: true
  - task: mv-nexus-license
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: ubuntu, tag: "trusty"}
      inputs:
        - name: git-nexus-license
        - name: git-nexus-docker-images
      run:
        path: /bin/sh
        args:
          - -c
          - mv -v git-nexus-license/nexus.lic git-nexus-docker-images/nexus.lic; ls -la git-nexus-docker-images
  - put: nexus-docker-image
    params:
      build: git-nexus-docker-images/

resources:
- name: git-nexus-docker-images
  type: git
  source:
    uri: [email protected]:dev/nexus-pro-dockerfile.git
    branch: test
    paths: [Dockerfile]
    private_key: {{git_ci_key}}

- name: git-nexus-license
  type: git
  source:
    uri: [email protected]:secrets/nexus-information.git
    branch: master
    paths: [nexus.lic]
    private_key: {{git_ci_key}}

- name: nexus-docker-image
  type: docker-image
  source:
    username: {{aws-token-username}}
    password: {{aws-token-password}}
    repository: {{ecr-nexus-repo}}

I've posted the pipeline that actually can be deployed to Concourse; however I tried a lot of things, but I can't figure out how to do this. I'm stuck on the part of moving the license file from git-nexus-license directory to git-nexus-docker-images directory. What I've done doesn't seem to mv the nexus.lic file because when while building the docker image it fails because it cannot find that file in the directory.

EDIT: I've successfully been able to "mv" nexus.lic using the code above, however the build is still failing due to not finding the file! I'm not sure what I'm doing wrong, the build works properly if I do it manually but with Concourse it's failing.


Solution

  • Okay so I figured out what I was doing wrong and as usual it was something small. I forgot to add the outputs to the yml file which tells concourse that this is the new workdir. Here's how it looks like now (which works for me):

    ---
    jobs:
    - name: build-nexus-docker-image
      public: false
      plan:
      - get: git-nexus-docker-images
        trigger: true
      - get: git-nexus-license
        trigger: true
      - task: mv-nexus-license
        config:
          platform: linux
          image_resource:
            type: docker-image
            source: {repository: ubuntu, tag: "trusty"}
          inputs:
            - name: git-nexus-license
            - name: git-nexus-docker-images
          outputs:
            - name: build-nexus-dir
          run:
            path: /bin/sh
            args:
              - -c
              - mv -v git-nexus-license/nexus.lic build-nexus-dir/nexus.lic; mv -v git-nexus-docker-images/* build-nexus-dir; ls -la build-nexus-dir;
      - put: nexus-docker-image
        params:
          build: build-nexus-dir/
    
    resources:
    - name: git-nexus-docker-images
      type: git
      source:
        uri: [email protected]:dev/nexus-pro-dockerfile.git
        branch: test
        paths: [Dockerfile]
        private_key: {{git_ci_key}}
    
    - name: git-nexus-license
      type: git
      source:
        uri: [email protected]:secrets/nexus-information.git
        branch: master
        paths: [nexus.lic]
        private_key: {{git_ci_key}}
    
    - name: nexus-docker-image
      type: docker-image
      source:
        username: {{aws-token-username}}
        password: {{aws-token-password}}
        repository: {{ecr-nexus-repo}}
    

    I hope this helps whoever gets stuck on this. :)