Search code examples
github-actionsgithub-actions-self-hosted-runners

Github actions: using two different kinds of self-hosted runners


I have a github repository which is doing CI/CD using github actions that need more than what the github-hosted runners can do. In multiple ways. For some tasks, we need to test CUDA code on GPUs. For some other tasks, we need lots of CPU cores and local disk.

Is it possible to route github actions to different self-hosted runners based on the task? Some tasks go to the GPU workers, and others to the big CPU workers? The docs imply this might be possible using "runner groups" but I honestly can't tell if this is something that A) can work if I figure it out B) will only work if I upgrade my paid github account to something pricier (even though it says it's "enterprise" already) or C) can never work.

When I try to set up a runner group following the docs, I don't see the UI elements that the docs describe. So maybe my account isn't expensive enough yet?

But I also don't see any way that I would route a task to a specific runner group. To use the self-hosted runners today, I just say

  gpu-test-job:
    runs-on: self-hosted

instead of

  standard-test-job:
    runs-on: ubuntu-22.04

and I'm not sure how I would even specify which runner group (or other routing mechanism) to get it to a specific kind of self-hosted runner, if that's even a thing. I'd need to specify something like:

  big-cpu-job:
    runs-on: self-hosted
    self-hosted-runner-group: big-cpu   # is this even a thing?

Solution

  • It looks like you won't be able to utilize runner groups on a personal account, but that's not a problem!

    Labels can be added to self-hosted runners. Those labels can be referenced in the runs-on value (as an array) to specify which self-hosted runner(s) the job should go to.

    You would run ./config.sh like this (you can pass in as many comma-separated labels as you like):

    ./config.sh --labels big-cpu
    

    and your job would use an array in the runs-on field to make sure it's selecting a self-hosted runner that is also has the big-cpu label:

    big-cpu-job:
        runs-on: [self-hosted, big-cpu]
        ...
    

    Note: If you wanted to "reserve" the big-cpu runners for the jobs that need it, then you'd use a separate label, regular, for example, on the other runners' ./config.sh and use that in the runs-on for the jobs that don't need the specialized runner.