BullMQ start Job from file with IIFE

Question

Let's say I have a function1.js (or ts, it doesn't matter in this case),function2.js and any other files which are IIFE with different logic, like that:

(async function F() {
  try {
    //[1,2,..n].map(x => console.log(x));
    //await any other action 
  } catch (e) {
    console.error(e)
  }
})()

and I have any job queue manager, in my case, it's BullMQ, but I guess it's relevant for Bull or Agenda. So the question is, can I put in the queue the files themselves?

So in the case of starting a new Worker, it will self-execute itself?

Like this:

const worker = new Worker(queueName, async (job: Job) => {
    // `path/to/functionN.js`
    return 'some value';
});

I understand that queue are created for the cases when one typical function requires args, which should be passed by MQ to do the routine tasks. But in this case, I have various function.js files and want to do managing the queue for them.

Answer 1

I have found a relative example in BullMQ docs. Actually, it's called sandbox processors, and only Bull and BullMQ, among any others job queue managers support such feature.

Other, you'll need to write your own implementation.

The alternative is using pm2 programmatic API which would manage files manually. In that case, you pass a direct path to file with IIFE, and then it has been executed via pm2.

    const worker = new Worker('Queue', async (job: Job) => {
      /**
       * job.data is path to file
       * not sure that await before pm2 is nessessary
       */
      console.log(job.data)
      await pm2.connect(err => {
        if (err) console.error(err)
        pm2.start({
          name: 'Task Name',
          script: job.data.path,
          exec_mode: 'cluster',
        }, (err) => {
          if (err) console.error(err)
          pm2.disconnect()
        });
      });
    }, {connection: connectionRedis});

In that case, you are able to monitor your task via pm2 -list command

If you are using TypeScript in your project, then don't forget that path should be to the compiled JS file, otherwise configure pm2 correctly