Search code examples
githubgithub-api

Is there a way to bulk/batch download all repos from Github based on a search result?


I run this search on Guthub and I get 881 repos. Blazor & C# repos. https://github.com/search?l=C%23&q=blazor&type=Repositories

Is there a way to download all these repos easily instead of one by one?


Solution

  • Yes, your query can be run via the github search api:

    That gives you one page of 100 repositories. You can loop over all pages, extract the ssh_url (or http if you prefer), and write the result to a file:

    # cheating knowing we currently have 9 pages
    for i in {1..9}
    do
        curl "https://api.github.com/search/repositories?q=blazor+language:C%23&per_page=100&page=$i" \
         | jq -r '.items[].ssh_url' >> urls.txt
    done
    
    cat urls.txt | xargs -P8 -L1 git clone
    

    You can optimize to extract the number of pages from the response headers.

    References:

    Similar question: