GitLab CI scheduled pipeline not running on default branch

I have a scheduled pipeline that copies some data from one server to another once a day.

The pipeline works fine on all branches but doesn't start if I select master branch (default branch) as the target branch.

I have tried on an exact copy of master and it worked fine.

I though it could be because master is protected but I tried on a protected copy of master and it worked.

I'm really not sure what's going on. When I click the "play" button next to the scheduled pipeline and it says that the job was successfully scheduled, but I cannot see any job in the job list.

Here some details on the .gitlab-ci.yml

stages:
  - copy_data
  - linting
  - test
  - deploy


lint:
  needs: []
  stage: linting
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH'
      changes:
        - "my_project/**/*.py"
  script:
    - ...

test:
  stage: test
  script:
    - ...
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH'

copy_database:on-schedule:
  stage: copy_data
  needs: []
  only:
    - schedules
    - $COPY_DB # this is set in the pipeline menu on gitlab
  script:
    - ...
  timeout: 3h

Solution

I finally figured out the problem! The issue is that gitlab was saying Successfully scheduled a pipeline to run. Go to the Pipelines page for details. while in reality there was an error.

To debug it I used the trick described here that is to run a manual pipeline and set CI_PIPELINE_SOURCE = "schedule". Running the pipeline in this way returned the error message and I was able to fix the issue.

If you are wondering what the error was, here is some more detail.

I had two pipelines optionally running on the master branch:

prepare_release:
  stage: prepare_release
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_TAG
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual
  script:
    - ...

create_release:
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  stage: release
  needs:
    - job: prepare_release
      artifacts: true
  rules:
     // HERE THERE SHOULD BE A RULE TO PREVENT THIS FROM RUNNING ON SCHEDULE
       // - if: $CI_PIPELINE_SOURCE == "schedule"
    //  when: never
    - if: $CI_COMMIT_TAG
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - ...

The second pipeline did not have a

    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

And therefore tried running on the schedule. However since it needs the previous one and that wasn’t created, it errors. Mi mistake was to assume that the "needs" would take into account this rule from the parent job.