Search code examples
javakotlinredisspring-data

Does redis pipelining guarantee repsonse order in cluster mode?


I use Spring Data Redis and Redis in cluster-mode. I want to check some parameters for existing.

        val results = redisTemplate.executePipelined { connection ->

        keyParams.forEach { params->
            val key = generateKey(params)
            connection.setCommands().sIsMember(key, valueToCheck(params))
        }
        null
    }

    return results
        .map { it as Boolean }

So, several questions:

  1. Is is correct for cluster mode if keys store in different nodes?
  2. Is redis guarantees repsonse order in cluster mode?

I searched a documentation but did'nt find the exactly answer


Solution

  • In order to use pipeline you will need all pipeline operations involve keys hashing into same slot. If you do so than yes, you will receive answer with same order of operations responses.

    1. You will have keys to hash to same slot in all operations you perform in pipeline
    2. Yes order is guaranteed

    My original answer to related topic: https://stackoverflow.com/a/72971327/5807561

    Copy below:

    Yes you can use pipeline in cluster mode as long as key hashes to same keyslot (not node).

    To achieve keys hashing to same slot you can use hash tags. TLDR - if you have curly brackets in your key {} hash function will only be applied to that part https://redis.io/docs/reference/cluster-spec/#hash-tags

    In my particular case I needed to put countries and cities by id into the redis which totaled 150k records. This is how I did it (nodejs + ioredis):

    if (countries.length && cities.length) {
      const pipelines = []
      for (let i = 0; i < 21; i++) {
        pipelines.push(redis.pipeline())
      }
      await Promise.all([
        Promise.map(countries, (country) => {
          const randomPipeline = pipelines[country.id % 21]
          randomPipeline.set(`country_${country.id}_{${country.id % 21}}`, JSON.stringify(country))
        }),
        Promise.map(cities, (city) => {
          const randomPipeline = pipelines[city.id % 21]
          randomPipeline.set(`city_${city.id}_{${city.id % 21}}`, JSON.stringify(city))
        })
        
    ])
      for (let i = 0; i < 21; i++) {
        await pipelines[i].exec()
      }
      console.log('Countries and cities in cache')
    }