Jira -- How to get issue changelog via REST API - but ALL, not single issue

I've seen this question many times, but no sufficient answer.

We're trying to dump all JIRA data into our data warehouse/ BI system. Or at least, the interesting parts.

One thing you can do is track status times, cycle time, lead time directly with field durations. This is very easy via JIRA's direct SQL database. The tables changeItem and changeGroup.

Of course the REST JSON API has less performance impact on the database.

However ... there appears to be no equivalent in the rest API of fetching ALL issue change history. Yes, you can fetch the changelog of one issue directly via an API call. If you have 100k issues, are you expected to make 100k API calls, iterating through issue IDs? Sounds like madness.

Is it somehow possible to expand changelogs through the search API, which amasses all issue data? I haven't seen it. Is what I'm seeking here possible? Or will we have to stick to the SQL route?

Solution

I think you are asking pretty much the same question as before: How can I fetch (via GET) all JIRA issues? Do I go to the Search node?, but additionally interesting in getting changelog data.

Yes, again you have to do it in batch, requesting JIRA API several times. Here is little bash script, which could help you to do that:

#!/usr/bin/env bash

LDAP_USERNAME='<username>'
LDAP_PASSWORD='<password>'

JIRA_URL='https://jira.example.com/rest/api/2/search?'
JQL_QUERY='project=FOOBAR'

START_AT=0
MAX_RESULTS=50

TOTAL=$(curl --silent -u "${LDAP_USERNAME}:${LDAP_PASSWORD}" -X GET -H "Content-Type: application/json" "${JIRA_URL}maxResults=0&jql=${JQL_QUERY}" | jq '.total')
echo "Query would export ${TOTAL} issues."


while [ ${START_AT} -lt ${TOTAL} ]; do
    echo "Exporting from ${START_AT} to $((START_AT + MAX_RESULTS))"
    curl --silent -u "${LDAP_USERNAME}:${LDAP_PASSWORD}" -X GET -H "Content-Type: application/json" "${JIRA_URL}maxResults=${MAX_RESULTS}&startAt=${START_AT}&jql=${JQL_QUERY}& expand=changelog" | jq  -c '.issues[]' >> issues.json

    START_AT=$((START_AT + MAX_RESULTS))
done

Please note the expand parameter, which additionally put all change log to the json dump as well. Alternatively you can use issue dumper python solution: implement the callback to store data to db and you're done.