Search code examples
bashyamlgithub-actions

How to clone multiple private repositories using GitHub Actions?

I have been searching a lot online for how to clone multiple private repositories while running a GitHub action script. Moreover, since the repositories I wish to clone are written in a text file within the repo itself, it complicates things a bit more for me. I mean, GitHub explains how to do this here: https://github.com/actions/checkout#checkout-multiple-repos-private but it assumes you know what you want to clone so you can list all the repos in the yml file. Also, they assume you have just one or two repos. What if you have 100 repos... I would rather use a script for that... So how to do that? Any Idea?

Solution

  • Summary:

    So you need to find a way to authenticate with github when you do the cloning. Then you do the cloning from a bash script that you can call from your yml file via github actions.

    Part1: Authentication:

    You can find in this link (https://dev.to/dtinth/authenticating-as-a-github-app-in-a-github-actions-workflow-27co) FOUR ways to authenticate and the pros and cons of each. Here is a summary of the methods:

    1. Method 1: Using the built-in GITHUB_TOKEN secret
    2. Method 2: Using your personal access token --> This is what I used with a small twist.
    3. Method 3: Creating a bot account and using its personal access token
    4. Method 4: Creating a GitHub App and generating tokens from it

    So the solution I used is Method 2 above in which I basically used my own PAT (Personal Access Token) to send to the bash script I wrote that does all the cloning for me. The nice thing about this is that I used the PAT as a secret and this way it is not exposed to anyone.

    Part2: Here is the part of the yml file that I used in github actions to do the cloning:

    - name: Run multi repo cloning script
        env: 
            PA_TOKEN: ${{ secrets.PAT_SECRET }} # `PAT_SECRET` is a secret that contains your PAT (Personal access token)
        run: ".github/clone_repos.sh"
        shell: bash

    Moreover, GitHub has a mechanism to detect GitHub tokens in the run logs when GitHub Actions run and if their mechanism detects a token it hides it with "***". So that is why there is very little risk for your token to be exposed by someone reviewing the GitHub Action output.

    Part3: in the bash script itself, I simply used the following command to clone all the repos I needed:

    #clone subrepo 
git clone "https://"$PA_TOKEN"@github.com/<remote_name>/"$SUBREPO_NAME".git"