Search code examples
node.jsredisjob-schedulingkue

How much Redis memory usage (approximately) is required to run 50 jobs using Bull and Kue?

I am trying to do an estimation of redis memory usage for Job scheduling (mostly spark jobs) and Bull and Kue are two options we are looking at this moment.

Solution

  • There is very little storage overhead to redis. With this small of a queue it may even compress your data. You just need to figure out what the job payload looks like. This will be dependent on the kind of job you are en queuing.

    In a normal scenario a job in the queue will have the name of the class being executed and a few parameters encoded in a string, often as JSON. This probably amounts to a few dozen bytes. Let's round way up and say 1KB per job to be safe.

    Size of job (<1KB) * Max Number of Jobs in Queue (50) = Required RAM (<50KB)

    Redis itself uses a few MB of RAM, so this amount is trivial. Adjust the above equation if your actual values are different.