Search code examples
pythonredisatomic

Redis list getting emptied without any reason?

I am consuming a redis list like this in python:

listitem = r.rpoplpush('mylist','mylist')

Strangely, the list gets empty randomly - for instance it will work without getting emptied for a month and then one fine day get emptied? What am I missing here? There is no other statement in my script which touches the script any way.

Solution

  • There is no such thing as an empty list in Redis, if a list is RPOP'ed all the way, the key is deleted.

    So, one of these should be happening:

    1. The list mylist is getting emptied (LPOP, RPOP, LREM, LTRIM, etc) all the way until empty.
    2. The key mylist is being deleted (DEL, UNLINK, etc)
    3. The key mylist is being expired (EXPIRE, EXPIREAT, etc)
    4. The key is being evicted
    5. Data loss is occurring

    If no redis-client is touching the key (1-3), then it must be 4 or 5.

    See if you have some eviction policy set in your server with CONFIG GET maxmemory-policy.

    Data loss may be occurring if you have no persistence and your server is restarted. Or if you are using more than a single instance (cluster or sentinel) and something is wrong. You can use the INFO command to see:

    • Server section: redis_mode and uptime_in_days.
    • Persistance section: relevant if uptime suggests the server restarted by the time you lost a key.
    • Memory & Stats sections: maxmemory_policy and evicted_keys will tell us if a policy is being applied
    • Replication & Cluster sections: if we still have no clue, start investigating this venue.