Running my Puppeteer app within PM2's cluster mode doesn't take advantage of the multiple processes

Question

While running my Puppeteer app with PM2's cluster mode enabled, during concurrent requests, only one of the processes seems to be utilized instead of all 4 (1 for each of my cores). Here's the basic flow of my program:

helpers.startChrome()
.then((resp) => {
    http.createServer(function (req, res) {
         const {webSocketUrl} = JSON.parse(resp.body);
         let browser = await puppeteer.connect({browserWSEndpoint: webSocketUrl}); 
         const page = await browser.newPage();

         ... //do puppeteer stuff

         await page.close();
         await browser.disconnect();
    })
})

and here is the startChrome() function:

    startChrome: function(){
        return new Promise(async (resolve, reject) => {
            const opts = {
                //chromeFlags: ["--no-sandbox", "--headless", "--use-gl=egl"],
                userDataDir: "D:/pupeteercache",
                output: 'json'
            };

            // Launch chrome using chrome-launcher.
            const chrome = await chromeLauncher.launch(opts);
            opts.port = chrome.port;

            // Connect to it using puppeteer.connect().
            resp = await util.promisify(request)(`http://localhost:${opts.port}/json/version`);
            resolve(resp);
        })
    }

First, I use a package called chrome-launcher to start up chrome, I then setup a simple http server that listens for incoming requests to my app. When a request is recieved, i connect to the chrome endpoint i setup through chrome-launcher at the beginning.

When i now try to run this app within PM2's cluster mode, 4 separate chrome tabs are opened up (not sure why it works this way but alright), and everything seems to be running fine. But when I send the server 10 concurrent requests to test and see if all processes are getting used, only the first one is. I know this because when i run PM2 monit, only the first process is using any memory.

Can someone explain to me why all the processes aren't utilized? Is it because of how i'm using chrome-launcher to only use one browser with multiple tabs instead of running multiple browsers?

Answer 1

You cannot use the same user directory for multiple instances at same time. If you pass a user directory, no matter what kind of launcher it is, it will automatically pick the running process and create a new tab on that instead.

Puppeteer creates a temporary profile whenever you want to launch the browser. So if you want to utilize 4 instances, pass it a different user data directory on each instance.