get a list of all our private repos

Question

I want to get a list of all private repositories in our organisation github. I tried -

curl -u user:pw -X GET  https://api.github.com/orgs/xxx/repos?per_page=100 >> list.txt 

I figure out that the per_page MAX value is 100 , how can I get all our repos into one file ?

Thanks

Answer 1

You can use a bash script that loops over each page until the last page reached is empty.

The filter can be done with jq JSON parser to filter private repository, you can also exclude forked repo or anything you want.

Note that I'm using a personal access token for the authentication (to get private repo) :

#!/bin/bash

# change those vars :
GITHUB_ORG=docker
GITHUB_ACCESS_TOKEN=12345666799897950400303332323
OUTPUT_FILE=repo_list.json

loop=0
index=1
TMP_FILE=tmpfile.txt
PER_PAGE=100

rm -f $TMP_FILE
echo "[]" > $OUTPUT_FILE

while [ "$loop" -ne 1 ]
do

    data=`curl -s "https://api.github.com/orgs/$GITHUB_ORG/repos?access_token=$GITHUB_ACCESS_TOKEN&page=$index&per_page=$PER_PAGE"`
    
    check_error=`echo "$data"  | jq 'type!="array"'`

    if [ "$check_error" == "true" ]; then
        echo "access token is invalid"
    exit 1
    fi

    filtered=`echo "$data" | jq '[ .[] | select(.private == true) ]'`

    if [ "$filtered" == "[]" ]; then
        loop=1
    else
        echo "$filtered" > $TMP_FILE
        concat=`jq -s add $TMP_FILE $OUTPUT_FILE`
        echo "$concat" > $OUTPUT_FILE
        size=`jq '. | length' $OUTPUT_FILE`
        echo "computed $index page - fetched total repo size of : $size"
        index=$((index+1))
    fi
done

If want to have only an array of repository URL instead of the whole JSON object, replace :

jq '[ .[] | select(.private == true)  ]'

with :

jq '[ .[] | select(.private == true)  | .html_url ]'