Search code examples
gitgithubcontinuous-integrationgithub-actionsparallel-testing

Where does self hosted github runner executes the code? Is having multiple runners on same machine a good practice?


Recently I have tried running the build test on my self hosted GitHub runners. One thing I know that it runs in its isolated virtual environment and destroys the environment at the end of the execution(Correct me if I am wrong). Now if we want to take control of the environment to debug how can we get access to this virtual environment?

Also to install multiple runners I have created multiple folders for multiple runners so that I can run my test in parallel using the same machine. Is there a chance that execution of one runner can interrupt execution on other runner if they are running on isolated virtual environments?


Solution

  • Where does self hosted github runner executes the code?

    One thing I know that it runs in its isolated virtual environment and destroys the environment at the end of the execution (Correct me if I am wrong)

    This is only true for GitHub-hosted runner but not for self-hosted runners. Self-hosted runners are implemented in a slightly different way in comparison to GitHub-hosted runners. According to the documentation

    each GitHub-hosted runner is always a clean isolated virtual machine, and it is destroyed at the end of the job execution

    Self-hosted runners are implemented as regular processes with no virtualization or isolation from the OS they are running on. It implies any side-effects caused by the GitHub action workflow job execution are visible to every process running in the OS. Any filesystem modifications are persisted as well across workflows' executions (eg. installing a dependency will be visible to other workflows being executed on the same runner later or even to the other runners set up on the same OS - like in your case). So you have to be aware once the runner is set up it's stateful between executing jobs. The fact implies some security drawbacks. You can read more about the differences between GitHub-hosted and self-hosted runners here.

    Now if we want to take control of the environment to debug how can we get access to this virtual environment?

    The basic debug capabilities are provided as several runner logging levels. If this is not enough you can also use a dedicated tmate debug action or search for the other debug-related actions you can find on theGitHub Marketplace

    Is there a chance that execution of one runner can interrupt execution on other runner if they are running on isolated virtual environments?

    As stated above, self-hosted runners do not use virtual environments but the existing OS environment. However, each runner set up on the same OS executes the workflow jobs independent from the other runner. Of course, a specific side-effect caused by the job in one runner can have a negative impact in the other. For example, if your integration tests being executed on one runner spin up a required http server on a given static port then the same tests will fail the http server set up on the other runner because that port will not be available.

    Is having multiple runners on same machine a good practice?

    As always, it depends. In general, there's nothing wrong with having multiple self-hosted runners being set up on the same OS. I used the approach on my own several times. But if you have concerns I mentioned about (the isolation, negative side-effects that may interfere between the job execution runs, security issues, etc.) then you should reconsider the approach.