Search code examples
javaredislettuce

mset over 400,000 map entries in redis


I'm using java with lettuce redis client. I have a local Redis instance and I'm reading a file and put 400,000 entries into a hash map. But after the end of the mset process there were no keys/values added into the redis store. I realized the problem is with the size of the map entries.

It's working when the map size is less than 50,000 entries.

Is there a way to increase this size or should I send this map as chunks?


Solution

  • Sending thousands of items at once doesn't sound healthy. Encoding 800000 items requires a huge buffer. Split up MSET into multiple commands if you can (1000 seems a reasonable size to me as a general rule of thumb, better: benchmark it!) or use transactions if you require atomicity.

    But after the end of the mset process there were no keys/values added into the redis store.

    Did any errors occur or did was the command completed successfully? Checking debug logs of Redis and Lettuce might help.