Search code examples
node.jspromiseqcancellation

Node promises: use first/any result (Q library)


I understand using the Q library it's easy to wait on a number of promises to complete, and then work with the list of values corresponding to those promise results:

Q.all([
    promise1,
    promise2,
    .
    .
    .
    promiseN,
]).then(results) {
    // results is a list of all the values from 1 to n
});

What happens, however, if I am only interested in the single, fastest-to-complete result? To give a use case: Say I am interested in examining a big list of files, and I'm content as soon as I find ANY file which contains the word "zimbabwe".

I can do it like this:

Q.all(fileNames.map(function(fileName) {
    return readFilePromise(fileName).then(function(fileContents) {
        return fileContents.contains('zimbabwe') ? fileContents : null;
    }));
})).then(function(results) {
    var zimbabweFile = results.filter(function(r) { return r !== null; })[0];
});

But I need to finish processing every file even if I've already found "zimbabwe". If I have a 2kb file containing "zimbabwe", and a 30tb file not containing "zimbabwe" (and suppose I'm reading files asynchronously) - that's dumb!

What I want to be able to do is get a value the moment any promise is satisfied:

Q.any(fileNames.map(function(fileName) {
    return readFilePromise(fileName).then(function(fileContents) {
        if (fileContents.contains('zimbabwe')) return fileContents;
        /*
        Indicate failure
        -Return "null" or "undefined"?
        -Throw error?
        */
    }));
})).then(function(result) {
    // Only one result!
    var zimbabweFile = result;
}).fail(function() { /* no "zimbabwe" found */ });

With this approach I won't be waiting on my 30tb file if "zimbabwe" is discovered in my 2kb file early on.

But there is no such thing as Q.any!

My question: How do I get this behaviour?

Important note: This should return without errors even if an error occurs in one of the inner promises.

Note: I know I could hack Q.all by throwing an error when I find the 1st valid value, but I'd prefer to avoid this.

Note: I know that Q.any-like behavior could be incorrect, or inappropriate in many cases. Please trust that I have a valid use-case!


Solution

  • You are mixing two separate issues: racing, and cancelling.

    Racing is easy, either using Promise.race, or the equivalent in your favorite promise library. If you prefer, you could write it yourself in about two lines:

    function race(promises) {
      return new Promise((resolve, reject) => 
        promises.forEach(promise => promise.then(resolve, reject)));
    }
    

    That will reject if any promise rejects. If instead you want to skip rejects, and only reject if all promises reject, then

    function race(promises) {
      let rejected = 0;
      return new Promise((resolve, reject) => 
        promises.forEach(promise => promise.then(resolve,
          () => { if (++rejected === promises.length) reject(); }
        );
    }
    

    Or, you could use the promise inversion trick with Promise.all, which I won't go into here.

    Your real problem is different--you apparently want to "cancel" the other promises when some other one resolves. For that, you will need additional, specialized machinery. The object that represents each segment of processing will need some way to ask it to terminate. Here's some pseudo-code:

    class Processor {
      promise() { ... }
      terminate() { ... }
    }
    

    Now you can write your version of race as

    function race(processors) {
      let rejected = 0;
      return new Promise((resolve, reject) => 
        processors.forEach(processor => processor.promise().then(
          () => {
            resolve();
            processors.forEach(processor => processor.terminate());
          },  
          () => { if (++rejected === processors.length) reject(); }
        );
      );
    }
    

    There are various proposals to handle promise cancellation which might make this easier when they are implemented in a few years.