Concurrency problem with Redis in Node.js

Question

Problem

I have a key called stocks in my Redis and its value is 1000. Suppose 300 customers request to buy (100 stocks each) at the same time. In the end, only 10 customers should be able to buy.

Solutions

I know it won't work but assume that my buy function is something like this:

/**
 * @param {import("redis").RedisClientType} instance
 * @param {string} clientId
 * @param {number} n
 */
async function buy(instance, clientId, n) {
  // --- (1) ----
  // Get current number of stocks
  let stocks = await instance.GET("stocks");
  stocks = parseInt(stocks);

  // --- (2) ----
  // Validate if the stocks remaining are enough to be bought
  if (stocks < n) {
    console.log("error: company does not have enough stocks");
    return new Error("error: company does not have enough stocks");
  }

  // --- (3) ----
  // Update the current stocks of the company and log who has bought stocks
  await instance.INCRBY("stocks", -n);
  console.log("client @%s has bought %s stocks successfully!", clientId, n);
}

To test it, I wrote a function that calls the buy function 300 times:

const redis = require("redis");
const crypto = require("crypto");
const { buy } = require("./buy");

async function main(customers = 300) {
  const instance = await redis
    .createClient({ url: "redis://localhost:6379" })
    .connect();

  // --- (1) ----
  // set stocks
  await instance.SET("stocks", 1000);

  // --- (2) ----
  // buy 100 stocks concurrentlly for each customer
  let pool = [];
  for (let i = 0; i < customers; i++) {
    let userId = crypto.randomBytes(4).toString("hex");
    pool.push(buy_v3(instance, userId, 100));
  }
  await Promise.all(pool);

  // --- (3) ----
  // Get the remaining stocks
  let shares = await instance.GET("stocks");
  console.log("the number of free shares the company has is: %s", shares);

  await instance.disconnect();
}
main();

Output:

...
error: company does not have enough stocks
error: company does not have enough stocks
error: company does not have enough stocks
error: company does not have enough stocks
the number of free stocks the company has is: -29000

As I said it didn't work but to fix that I used this approach:

/**
 * @param {import("redis").RedisClientType} instance
 * @param {string} clientId
 * @param {number} n
 */
async function buy(instance, clientId, n) {
  try {
    await instance.executeIsolated(async (client) => {
      // --- (1) ----
      // Get current number of stocks
      let stocks = await client.GET("stocks");
      stocks = parseInt(stocks);

      // --- (2) ----
      // Validate if the stocks remaining are enough to be bought
      if (stocks < n) {
        throw new Error("error: company does not have enough stocks");
      }

      // --- (3) ----
      // Update the current stocks of the company
      await client.INCRBY("stocks", -n);
    });

    console.log("client @%s has bought %s stocks successfully!", clientId, n);
  } catch (err) {
    console.log(err.message);
  }
}

And if you test it again you will see something like this:

...
error: company does not have enough stocks
error: company does not have enough stocks
error: company does not have enough stocks
error: company does not have enough stocks
the number of free stocks the company has is: 0

That means it works without a problem.

Question

The above solution works well but I'm a little confused about executeIsolated function. As far as I know, it just creates a new connection (you can see here) and it's useful when you want to run your commands on an exclusive connection like watch command.

Who can explain what is the exact role of executeIsolated in my case?

Answer 1

The issue is that when there are concurrent requests there is no guarantee that the SET of the nth request will run before the GET of the nth+1 request. For example, in case there are 2 concurrent requests, the commands should be executed in this order:

> GET stocks
"100"
> INCRBY stocks -100
(integer) 0
> GET stocks
"0"

but they might be executed in this order:

> GET stocks
"100"
> GET stocks
"100"
> INCRBY stocks -100
(integer) 0
> INCRBY stocks -100
(integer) -100

In order to fix it you should use a Redis function (available since redis 7.0) or Lua Script that looks something like this:

local stocks = redis.call('GET', KEYS[1])
if stocks < ARGS[1] then
  return redis.error_reply('company does not have enough stocks')
end

redis.call('SET', KEYS[1], stocks - ARGS[1])
return redis.status_reply('OK')

Regarding why the issue was "fixed" using executeIsolated - there could be 2 reasons for that:

1. the pool size is 1, which effectively creates a queue
2. there were no "idle" connections in the pool, and the time it takes to create a new connection is more than the time it takes to execute GET..