classic problem, this time encountered in a nodejs environment and after thinking about it for a while i'm not sure what's the best way to solve this.
i have:
A combination of 1 task and 1 resource (doesn't matter which) is a job to be done by a worker: or in other words - worker opens the lock with the key:)
I want to open to open these locks as fast as possible using my resources and workers.
Each worker should try to grab a key, grab a lock, unlock the lock, then put his key back.
When no keys are available a worker needs to wait till one is there.
When all locks are open workers can go home and enjoy a beer.
how would you solve this in a javascript / nodejs environment?
feel free to use redis/mongo/whatever tool you need to make this happen.
Please help me set my workers early back home today! :)
You can use RabbitMQ to solve this problem. You can maintain 4 different queues for each worker. Every worker will be bind to a separate queue.
So the worker will be waiting till there is no message on the queue. As soon as there is a message in your case a key the worker will start processing complete the task and again wait for the next key.
There will be single publisher and multiple listeners i.e. single supervisor and multiple workers. You can even implement a batch processing if required.
for example:
connection.on('ready', function () {
connection.queue("queue_name", function(queue){
connection.exchange('exchange_name', {type: 'topic', confirm: true},function(exchange){
queue.bind('exchange_name');
queue.subscribe({ack: true},function (message, headers, deliveryInfo) {
try{
var data = unescape(message.data);
processMessage(data); // whatever needs to be done with the received message
queue.shift();
}
catch(e){
console.log('Some error occured.'+ e);
}
})
})
})
})