How to use yield with callback based loops?

Question

Although the primary purpose of the yield keyword is to provide iterators over some data, it is also rather convenient to use it to create asynchronous loops:

function* bigLoop() {
    // Some nested loops
    for( ... ) {
        for( ... ) {
            // Yields current progress, eg. when parsing file
            // or processing an image                        
            yield percentCompleted;
        }
    }
}

This can then be called asynchronously:

function big_loop_async(delay) {
    var iterator = big_loop();
    function doNext() {
        var next = iterator.next();
        var percent_done = next.done?100:next.value;
        console.log(percent_done, " % done.");
        // start next iteration after delay, allowing other events to be processed
        if(!next.done)
            setTimeout(doNext, delay);
    }
    setTimeout(doNext, delay);
}

However, in modern javascript, callback based loops have become quite popular. We have Array.prototype.forEach, Array.prototype.find or Array.prototype.sort. All of these are based on a callback passed for each iteration. I even heard it to be recommended that we use these if we can, because they can be optimized better than standard for loops.

I also often use callback based loops to abstract away some complex looping pattern.

And the question here is, is it possible to turn those into yield based iterators? As a simple example, consider I wanted you to sort an array asynchronously.

Answer 1

tl;dr: You can’t do that, but check out this other thing you can do with the latest V8 and bluebird:

async function asyncReduce() {
    const sum = await Promise.reduce(
        [1, 2, 3, 4, 5],
        async (m, n) => m + await Promise.delay(200, n),
        0
    );

    console.log(sum);
}

No, it’s not possible to make Array.prototype.sort accept the results of comparisons asynchronously from its comparison function; you would have to reimplement it entirely. For other individual cases, there might be hacks, like a coroutiney forEach (which doesn’t even necessarily work as you’d expect, because every generator will run until its first yield before one continues from a yield):

function syncForEach() {
    [1, 2, 3, 4, 5].forEach(function (x) {
        console.log(x);
    });
}

function delayed(x) {
    return new Promise(resolve => {
        setTimeout(() => resolve(x), Math.random() * 1000 | 0);
    });
}

function* chain(iterators) {
    for (const it of iterators) {
        yield* it;
    }
}

function* asyncForEach() {
    yield* chain(
        [1, 2, 3, 4, 5].map(function* (x) {
            console.log(yield delayed(x));
        })
    );
}

and reduce, which works nicely by nature (until you look at performance):

function syncReduce() {
    const sum = [1, 2, 3, 4, 5].reduce(function (m, n) {
        return m + n;
    }, 0);

    console.log(sum);
}

function* asyncReduce() {
    const sum = yield* [1, 2, 3, 4, 5].reduce(function* (m, n) {
        return (yield* m) + (yield delayed(n));
    }, function* () { return 0; }());

    console.log(sum);
}

but yeah, no magic wand for all functions.

Ideally, you’d add alternate promise-based implementations for all these functions – popular promise libraries, like bluebird, already do this for map and reduce, for example – and use async/await instead of generators (because async functions return promises):

async function asyncReduce() {
    const sum = await Promise.reduce(
        [1, 2, 3, 4, 5],
        async (m, n) => m + await delayed(n),
        0
    );

    console.log(sum);
}

You wouldn’t need to wait for async support to do this so much if ECMAScript had sane decorators like Python, either:

@Promise.coroutine
function* add(m, n) {
    return m + (yield delayed(n));
}

@Promise.coroutine
function* asyncReduce() {
    const sum = yield Promise.reduce([1, 2, 3, 4, 5], add, 0);

    console.log(sum);
}

… but it doesn’t and so you do. Or you can live with code like this:

const asyncReduce = Promise.coroutine(function* () {
    const sum = yield Promise.reduce([1, 2, 3, 4, 5], Promise.coroutine(function* (m, n) {
        return m + (yield delayed(n));
    }), 0);

    console.log(sum);
});