Call stack size exceeded on re-starting Node function

Question

I'm trying to overcome Call stack size exceeded error but with no luck,

Goal is to re-run the GET request as long as I get music in type field.

//tech: node.js + mongoose
//import components
const https = require('https');
const options = new URL('https://www.boredapi.com/api/activity');


//obtain data using GET
https.get(options, (response) => {
  //console.log('statusCode:', response.statusCode);
  //console.log('headers:', response.headers);

  response.on('data', (data) => {
        //process.stdout.write(data);
        apiResult = JSON.parse(data);
        apiResultType = apiResult.type;
        returnDataOutside(data);
  });
 })

.on('error', (error) => {
  console.error(error);
 });


function returnDataOutside(data){
  apiResultType;

  if (apiResultType == 'music') {
    console.log(apiResult);
  } else {
    returnDataOutside(data);
    console.log(apiResult); //Maximum call stack size exceeded
  };
};

Answer 1

Your function returnDataOutside() is calling itself recursively. If it doesn't gets an apiResultType of 'music' on the first time, then it just keeps calling itself deeper and deeper until the stack overflows with no chance of ever getting the music type because you're just calling it with the same data over and over.

It appears that you want to rerun the GET request when you don't have music type, but your code is not doing that - it's just calling your response function over and over. So, instead, you need to put the code that makes the GET request into a function and call that new function that actually makes a fresh GET request when the apiResultType isn't what you want.

In addition, you shouldn't code something like this that keeping going forever hammering some server. You should have either a maximum number of times you try or a timer back-off or both.

And, you can't just assume that response.on('data', ...) contains a perfectly formed piece of JSON. If the data is anything but very small, then the data may arrive in any arbitrary sized chunks. It make take multiple data events to get your entire payload. And, this may work on fast networks, but not on slow networks or through some proxies, but not others. Instead, you have to accumulate the data from the entire response (all the data events that occur) concatenated together and then process that final result on the end event.

While, you can code the plain https.get() to collect all the results for you (there's an example of that right in the doc here), it's a lot easier to just use a higher level library that brings support for a bunch of useful things.

My favorite library to use in this regard is got(), but there's a list of alternatives here and you can find the one you like. Not only do these libraries accumulate the entire request for you with you writing any extra code, but they are promise-based which makes the asynchronous coding easier and they also automatically check status code results for you, follow redirects, etc... - many things you would want an http request library to "just handle" for you.