Optimal way of sharing load between worker threads

Question

What is the optimal way of sharing linear tasks between worker threads to improve performance?

Take the following example of a basic Deno web-server:

Main Thread

// Create an array of four worker threads
const workers = new Array<Worker>(4).fill(
    new Worker(new URL("./worker.ts", import.meta.url).href, {
        type: "module",
    })
);

for await (const req of server) {
    // Pass this request to worker a worker thread
}

worker.ts

self.onmessage = async (req) => {
  //Peform some linear task on the request and make a response
};

Would the optimal way of distributing tasks be something along the lines of this?

function* generator(): Generator<number> {
    let i = 0;
    while (true) {
        i == 3 ? (i = 0) : i++;
        yield i;
    }
}

const gen = generator();

const workers = new Array<Worker>(4).fill(
    new Worker(new URL("./worker.ts", import.meta.url).href, {
        type: "module",
    })
);

for await (const req of server) {
    // Pass this request to a worker thread
    workers[gen.next().value].postMessage(req);
}

Or is there a better way of doing this? Say, for example, using Attomics to determine which threads are free to accept another task.

Answer 1

When working with WorkerThread code like this, I found that the best way to distribute jobs was to have the WorkerThread ask the main thread for a job when the WorkerThread knew that it was done with the prior job. The main thread could then send it a new job in response to that message.

In the main thread, I maintained a queue of jobs and a queue of WorkerThreads waiting for a job. If the job queue was empty, then the WorkerThread queue would likely have some workerThreads in it waiting for a job. Then, any time a job is added to the job queue, the code checks to see if there's a workerThread waiting and, if so, removes it from the queue and sends it the next job.

Anytime a workerThread sends a message indicating it is ready for the next job, then we check the job queue. If there's a job there, it is removed and sent to that worker. If not, the worker is added to the WorkerThread queue.

This whole bit of logic was very clean, did not need atomics or shared memory (because everything was gated through the event loop of the main process) and wasn't very much code.

I arrived at this mechanism after trying several other ways that each had their own problems. In one case, I had concurrency issues, in another I was starving the event loop, in another, I didn't have proper flow control to the WorkerThreads and was overwhelming them and not distributing load equally.