Search code examples
node.jsredisredis-search

Redis Aggregation using NodeJS

I have the hash data pushed to redis and I want to apply group by on the data and get the results

Assume I have the following data on redis as Hash with the key '3801'

HSET "3801:1" username "albert" pids "11,12"
HSET "3801:2" username "bob" pids "11,12"
HSET "3801:3" username "casey" pids "13,14"
HSET "3801:4" username "david" pids "13"

I want to create index and group them based on pids. So, when I pass the input as 3801, I need the following as result (within the key)

albert,bob
casey
david

Redis Create Command:

FT.CREATE 3801:idx ON HASH PREFIX 1 3801: SCHEMA username TEXT SORTABLE pids TEXT SORTABLE

Redis Aggregate Command:

FT.AGGREGATE 3801:idx '*' GROUPBY 1 '@pids' REDUCE TOLIST 1 '@username' AS 'users'

These redis commands worked just fine.

NodeJs code I have used to create the index

redisClient.ft.create('3801:idx', {
      username: {
        type: SchemaFieldTypes.TEXT,
        SORTABLE: true,
      },
      pids: {
        type: SchemaFieldTypes.TEXT,
        SORTABLE: true,
      },
    }, {
      ON: 'HASH',
      PREFIX: '3801:',
    })

The generated create redis command from logs:

"FT.CREATE" "3801:idx" "ON" "HASH" "PREFIX" "1" "3801:" "SCHEMA" "username" "TEXT" "SORTABLE" "pids" "TEXT" "SORTABLE"

and for aggregation,

const result = await new Promise((resolve, reject) => {
    redisClient.ft.aggregate('3801:idx', '*', {
      STEPS: [{
        type: AggregateSteps.GROUPBY,
        properties: ['@pids'],
        REDUCE: [{
          type: AggregateGroupByReducers.TOLIST,
          property: '@username',
          AS: 'users',
        }],
      }],
    }, (err, result) => {
      if (err) {
        reject(err)
      } else {
        resolve(result)
      }
    })
  })
  console.log('got result:', JSON.stringify(result))

This code generated the following

"FT.AGGREGATE" "3801:idx" "*" "GROUPBY" "1" "@pids" "REDUCE" "TOLIST" "1" "@username" "AS" "users"

Looks like the commands generated are correct. I just confirmed it by copying this to redis-cli and ran. I got the expected result. But the NodeJS code not returning the result

Screenshot from RedisInsight on the data

enter image description here

Can someone please help me on how can I solve this using NodeJs?

Solution

  • I just removed the Promise handler which was surrounded by the aggregate call and it worked just fine

    const result = await redisClient.ft.aggregate(indexName, queryPattern, {
      STEPS: [{
        type: AggregateSteps.FILTER,
        expression: `${userList.map(username => `@username=='${username}'`).join(' || ')}`,
      }, {
        type: AggregateSteps.GROUPBY,
        properties: ['@pids'],
        REDUCE: [{
          type: AggregateGroupByReducers.TOLIST,
          property: '@username',
          AS: 'users',
        }],
      }],
    })
    logger.info('Got result\nSize: ', JSON.stringify(result.total))
    logger.debug('\nData: ' + JSON.stringify(result.results))
  } catch (error) {
    logger.error('Error:', error)
  }