How to properly loop through HTTP requests using XMLHttpRequest()?

I am currently learning vanilla JS before jumping into any major frameworks so I can get the grip with HTTP request mechanics. I am sure there are a ton of better JS libraries for HTTP requests but I want to figure out how to solve my problem using classic XMLHttpRequest(), therefore please do not suggest any solutions that do not include XMLHttpRequest().

I am trying to loop through an array I have and make HTTP GET requests to an API with the information in the array and then populate my HTML with the response data. The code is quite simple:

Function that takes in my array of numbers:

function loadUniqueProjs(uniqueArray)
{

    var reqObject = [];

    for (var i in unique_array) 
    {
         outsideLoopRequest(unique_array[i], reqObject, i);
    }


}

I loop through the array and execute a function that is supposed to execute my GET request:

function outsideLoopRequest(arrayValue,reqObject, i){

    // Create XHR Object
    reqObject[i] = new XMLHttpRequest();
    reqObject[i].open('GET', 'http://localhost:3000/resource/Projects/' + arrayValue, true)

    reqObject[i].onload = function() {
        if (this.status == 200) {
            var client = JSON.parse(this.responseText);
            var output = '';

            for (var j in client){

                output +=   '<div class="card">'+
                                    '<h5 class="card-header" role="tab" id="heading'+ j + '">'+
                                        '<a data-toggle="collapse" data-parent="#accordion" style="color:black"' + 
                                            'href="#collapse' + j + '"aria-expanded="false" aria-controls="collapse' + j + '"class="d-block collapsed">' +
                                            '<i class="fa fa-angle-double-down pull-right"></i>#' +
                                            client[j].projectTitle + ' | ' + 'PM: ' + client[j].projectManager + ' | ' +
                                            'PO: ' + client[j].projectOwner + ' | ' + 'Estimated Deadline: ' + client[j].predictedCompletion +
                                            ' | ' + 'Status: ' + client[j].status +
                                            ' | ' + 'Requestor: ' + client[j].requestor +
                                        '</a>'+
                                    '</h5>'+
                            '</div>';
            }
    }

    document.getElementById("spinner").hidden = true;
    // output the data into the HTML page
    document.getElementById('accordion').innerHTML = output;

    console.log(this.status + ' Data retrieved successfully!');

}
    reqObject[i].send();
}

After hours of step by step debugging I learned that the HTTP requests are asynchronous and when utilising a loop to execute the requests one by one they won't behave like you want them to. The requests do not get executed one by one adding the HTML I require, instead the loop opens all requests first and then executes as they come, and when stepping through code in the web debugger the code jumps around all over the place it gets extremely confusing (sorry for rant).

I would like it to behave step by step. I did research on SO and someone suggested that these are scoping issues and that the requests should be made in a separate function, which is why I have structured my code with the loop in one function and the request execution in another but its still missbehaving as described previously. Is there anyone that could share their expertise please?

Solution

The only way to do something after XMLHttpRequest has completed is to pass that "something" as a callback. For a chain of request, you'll have to do it recursively, so the first one will have to receive an ordered list of all the requests to execute after plus the finishing callback to do after the final one is completed. The code will probably get ugly, I'm not going to try.
You can still send all your requests right away and render the data in the correct order as they come. Start with creating the basic skeleton structure of the output data

var outputStructure = '';

for (var i in unique_array) {
   var cardId = 'card-' + id;
   outputStructure += `<div class="card" id="card-${i}">`;
   outsideLoopRequest(unique_array[i], reqObject, i);
}

document.getElementById('accordion').innerHTML = outputStructure;

and on completion, put the data in the card with the right id.

    reqObject[i].onload = function() {
// create the output structure, then
       document.getElementById(`card-${i}`).innerHTML = output;

Problems as such are a part of what is colloquially called "callback hell". It's simply so much easier to organize more complicated async operations (where some things can be done in parallel, others must wait until the previous one has completed) with promises. Do stuff after all requests have been completed? One line: Promise.all(requests).then(//....

Use fetch.