I would like to implement a concurrent priority queue in Redis, with multiple processes on different machines adding items (with scores) and multiple other processes popping these items, lowest score first.
A simple queue can be implemented with LPUSH and RPOP.
Using a ZSET, I can add the items using ZADD and pop them with ZRANGE and ZREM, as long as there is only one reader.
For multiple readers I think I need something like ZPOP which combines ZRANGE and ZREM in a single atomic operation. Otherwise two readers may get the same item from ZRANGE before either can ZREM it. Retrying if ZREM returns 0 would work but is not desirable.
Is there some way I can do this using the current Redis commands? Is there any reason this hasn't been added to Redis already? It seems like it would be a pretty simple command to implement.
You can guarantee atomicity if you use a Lua script that does the ZRANGE & ZREM or with a MULTI/EXEC block. This will prevent multiple workers from interfering with each other.
I assume that ZPOP wasn't put in in the first place because it isn't a common use case and, when needed, it can be easily scripted.