Search code examples
npmgitlab-cigitlab-ci-runnercicd

GitLab CICD pipeline not sharing artifcacts between stages


I have a very simple GitLab CICD pipeline with build and deploy stages. However, the build artifact is not being retained for deploy, and I cannot figure out how to get the dist directoty to be reatined. YAML is shown below:

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy

build:
  image: node
  script:
    - npm install
    - npm run build
    - pwd; ls -latrR dist
  artifacts:
    paths:
      - "dist/"
    expire_in: 1 week

deploy:
  dependencies:
    - "build"
  script:
    - pwd; ls -latrR
    - apt-get update -qq && apt-get install -y -qq lftp
    - if [ -d 'dist' ] ; then lftp -c "set ftp:ssl-allow no; open -u $FTP_USERNAME,$FTP_PASSWORD $FTP_HOST; mirror -v dist/ $FTP_DESTINATION --reverse --ignore-time --parallel=10 --exclude-glob .git* --exclude .git/"; else echo "No dist directory"; fi;
  environment:
    name: production
  only:
    - main

The build pipeline stage seems to be doing cleanup even though expire_in is set

Uploading artifacts...
dist/: found 45 matching artifact files and directories 
WARNING: Upload request redirected                  location=https://gitlab.com/api/v4/jobs/7052630492/artifacts?artifact_format=zip&artifact_type=archive&expire_in=1+week new-url=https://gitlab.com
WARNING: Retrying...                                context=artifacts-uploader error=request redirected
Uploading artifacts as "archive" to coordinator... 201 Created  id=7052630492 responseStatus=201 Created token=glcbt-65
Cleaning up project directory and file based variables
00:00
Job succeeded

The recursive listing and pwd command shows that the dist directory is absent from the deploy stage. This can be shown below from the pipeline output:

$ apt-get update -qq && apt-get install -y -qq lftp
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package lftp.
(Reading database ... 23248 files and directories currently installed.)
Preparing to unpack .../lftp_4.9.2-2+b1_amd64.deb ...
Unpacking lftp (4.9.2-2+b1) ...
Setting up lftp (4.9.2-2+b1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
$ if [ -d 'dist' ] ; then lftp -c "set ftp:ssl-allow no; open -u $FTP_USERNAME,$FTP_PASSWORD $FTP_HOST; mirror -v dist/ $FTP_DESTINATION --reverse --ignore-time --parallel=10 --exclude-glob .git* --exclude .git/"; else echo "No dist directory"; fi;
No dist directory
Cleaning up project directory and file based variables
00:00
Job succeeded

Any assistance would be gratefully received,

Cheers Andrew


Solution

  • Figured out that the job names need to be separate from the stage names. Working YAML below:

    stages:          # List of stages for jobs, and their order of execution
      - build
      - test
      - deploy
    
    prod-build:
      stage: build
      image: node
      script:
        - npm install
        - npm run build
      artifacts:
        name: 'dist'
        paths:
          - 'dist/'
        expire_in: 1 week
      rules:
        - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    
    prod-test:
      stage: test
      dependencies:
        - 'prod-build'
      script:
        - echo "Tests TBC"
      rules:
        - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    
    prod-deploy-tr4e:
      stage: deploy
      image: mwienk/docker-lftp
      rules:
        - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      dependencies:
        - 'prod-build'
      before_script:
        - pwd
        - ls -latrR dist # list contents of the dist directory
        - echo "Uploading to " ${FTP_HOST} " under " ${FTP_DESTINATION}; # dump target SFTP host and directory
      script: # Check that the dist directory exists from the build stage
        - |
          if [ -d 'dist' ] ; then 
            cd dist; 
            lftp -c "set ftp:ssl-allow no; open -u $FTP_USERNAME,$FTP_PASSWORD $FTP_HOST; mirror -v ./ $FTP_DESTINATION --reverse --ignore-time --parallel=10 --exclude-glob .git* --exclude .git/"
          else 
            echo "Missing dist directory"
          fi