Search code examples
redisredis-py

Finding value collisions from a changing collection of Redis keys


In my website, users are allowed to keep the same usernames. Moreover, at any point in time a user logs in, I temporarily save their username in a redis key with a ttl of 10 mins.

The question is: is there any way - using Redis - to find all user ids online within the last 10 mins, sharing the same username?

Currently, I'm extracting all the keys' values and finding collisions in Python - which doesn't really help since I need to do this multiple times at runtime (and there's lots of user traffic).

I hypothesize that I could have created sets with a unique username as the key, and stored all user ids in the set to give me O(1) look ups on users sharing the same usernames. But then, I'd have to sacrifice the 10 mins ttl condition (which I need for every username individually).

Btw Redis/Lua beginner here, hence the noob question (if it is).


Solution

  • Where there is a will, there is a way... :)

    Begin by storing the logins in a Sorted Set. Assuming that user id 123 had logged in at time 456 with the username "foo", you can represent that as:

    ZADD logins 456 123:foo
    

    Note: you'll also have to remove old elements from that Sorted Set so it doesn't just grow out of control.

    Next, you want to search for the users from the last 10 minutes, so you'd use ZRANGEBYSCORE for that. Instead of shipping the entire thing back to your client, use Lua to process it and check for collisions.

    The following script example wraps together all of the above:

    -- Keys: 1) The logins Sorted Set
    -- Args: 1) The epoch value of 'now'
    --       2) The logged in user id
    --       3) The logged in user name         
    
    -- Get logins from the last 10 minutes
    local l = redis.call('ZRANGEBYSCORE', KEYS[1], ARGV[1]-600, '+inf')
    
    -- "Evict" old logins
    redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', '(' .. ARGV[1]-600)
    
    -- Store the new login
    redis.call('ZADD', KEYS[1], ARGV[1], ARGV[2] .. ':' .. ARGV[3])
    
    local c = {}  -- detected name collision
    for _, v in pairs(l) do
      local p = v:find(':')  -- no string.split in Lua
      local i = v:sub(1,p-1) -- id
      local n = v:sub(p+1)   -- name
      if n == ARGV[3] then
        c[#c+1] = i
      end
    end
    
    return c