how to trigger new job by comment in Pull Request for GitHub Actions

Question

Let's say that I have a workflow:

---
name: 'DEV Test'

on:
  issue_comment:
    types: [created]
  pull_request:
    branches:
      - 'develop'

defaults:
  run:
    shell: python

jobs:
  test:
    # yamllint disable rule:line-length
    if: ${{ (github.event_name == 'issue_comment' && github.event.issue.pull_request && startsWith(github.event.comment.body, 'continue_even_if_fail')) || github.event_name == 'pull_request' }}
    name: 'DEV Test'
    runs-on: ubuntu-latest
    steps:
      - name: Check if specific directives exists
        env:
          PR_MSG: ${{ github.event.pull_request.body }}
          PR_COMMENT: ${{ github.event.comment.body }}
        run: |
          from os import environ

          global_envs_file = environ.get('GITHUB_ENV')

          # if event type is comment get message from comment body
          if '${{ github.event_name }}' == 'issue_comment':
            msg = environ.get('PR_COMMENT', '')
          # if event type is PR create or sync get message from PR message
          else:
            msg = environ.get('PR_MSG', '')

          # set global ENV `CONTINUE` to 0 or 1 depending on 'continue_even_if_fail' existance
          with open(global_envs_file, 'a') as f:
            if 'continue_even_if_fail' in msg:
              f.write('CONTINUE=1\n')
            else:
              f.write('CONTINUE=0\n')

      - name: fail
        if: ${{ env.CONTINUE == 0 }}
        uses: actions/github-script@v3
        with:
          script: |
              core.setFailed('Set to fail')

      - name: complete
        run: |
          print('''
          ${{ toJSON(github) }}
          ''')

So if we have continue_even_if_fail identifier in message or PR comment, workflow should end successfully in theory. In other way it will fail (again in theory)

＊It is just a synthetic simplified example and actual logic is more complicated and meaningful

Problem

This workflow works correctly if I update or create Pull Request. But nothing happens if I add comment to PR conversation.

Question

How to trigger test job in PR that will successfully end by new specific comment with continue_even_if_fail message

PS

even if I simplify condition

from

if: ${{ (github.event_name == 'issue_comment' && github.event.issue.pull_request && startsWith(github.event.comment.body, 'continue_even_if_fail')) || github.event_name == 'pull_request' }}

to

if: ${{ github.event_name == 'issue_comment' || github.event_name == 'pull_request' }}

Nothing will be triggered

Documentation that I used to create the workflow: about events, about triggers, especially about issue_comment trigger, about issue_comment in Japanese

Answer 1

The problem was obvious but not intuitive at same time. So, I decided to share cause as answer.

Let's imagine that you have three branches: feature, develop, main. Where main is a default branch.
And now you trying to do Pull Request feature → develop.

The import point is that issue_comment is always triggered under default branch (main in my case). But pull_request is triggered under Pull Request's branch (merged version of feature and develop in my case)

It means that run_test.yaml workflow should exists in default branch main to be triggered by issue_comment. In other way pull_request will be triggered if run_test.yaml workflow exists in feature or develop branches.

To be able trigger both actions issue_comment and pull_request we have to add run_test.yaml file at main and feature (develop) branches first.

And that was cause in my case, because workflow file was newly created inside Pull Request and only pull_request event was triggered.

Here is a caution point: Because triggered branch is main you have to manually set PR's branch when checkout:

steps:
    - name: "Get PR's branch name"
      id: get_branch
      run: |
        PR=$(curl -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" ${{ github.event.issue.pull_request.url }})
        echo "::set-output name=branch::$(echo $PR | jq -r '.head.ref')"

    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        ref: ${{ steps.get_branch.outputs.branch }}

And here is a second problem why issue_comment can't be used as Pull Request internal trigger for CI tests: because it triggered under default main branch it runs outside of the Pull Request as separated Action. And even if it completes with success PR's last CI test still will be marked as failed (with × mark)