Question: I'm trying to figure out how to get the number of pending jobs/builds/pipelines from the GitLab API.
Background: The goal is to use this data as the base for auto-scaling the CI runners based on pending buils. Due to the spiky nature of our jobs regular CPU-based autoscaling doesn't work well. The jobs finish too fast and the runners take too long to start up.
What I've done: I've looked at the Pipelines API as well as the Jobs API. We're polling all jobs with Scope pending and all Pipelines with Status pending, but these are always zero. Even if there are pending builds.
TBH I'm not even sure about the terminology here. Should I be looking at the Jobs API, the Pipelines API or something else altogether?
Also I'm not sure about the meaning of Scope vs. Status here. Unfortunately it isn't explained in the API docs.
Any help would be appreciated.
You should get the list of pending jobs from Jobs API:
curl -g --header "PRIVATE-TOKEN: <your_private_token>" "https://<gitlab_url>/api/v4/projects/<project_id>/jobs?scope=pending"
Example: I start a pipeline where is 9 jobs, but the concurrency on my runner is set to 3 :
curl -g --header "PRIVATE-TOKEN: <your_private_token>" "https://<gitlab_url>/api/v4/projects/<project_id>/jobs?scope=pending" | jq '. | length'
And output is correctly 6.