How to prevent race condition when using redis to implement flow control?

Question

We have a server that gets cranky if it gets too many users logging in at the same time (meaning less than 7 seconds apart). Once the users are logged in, there is no problem (one or two logging in at the same time is also not a problem, but when 10-20 try the entire server goes into a death spiral sigh).

I'm attempting to write a page that will hold onto users (displaying an animated countdown etc.) and let them through 7 seconds apart. The algorithm is simple

1. fetch the timestamp (t) when the last login happened
2. if t+7 is in the past start the login and store now() as the new timestamp
3. if t+7 is in the future, store it as the new timestamp, wait until t+7, then start the login.

A straight forward python/redis implementation would be:

import time, redis
SLOT_LENGTH = 7  # seconds

now = time.time()

r = redis.StrictRedis()

# lines below contain race condition..
last_start = float(r.get('FLOWCONTROL') or '0.0')  # 0.0 == time-before-time
my_start = last_start + SLOT_LENGTH
r.set('FLOWCONTROL', max(my_start, now))  

wait_period = max(0, my_start - now)
time.sleep(wait_period)

# .. login

The race condition here is obvious, many processes can be at the my_start = line simultaneously. How can I solve this using redis?

I've tried the redis-py pipeline functionality, but of course that doesn't get an actual value until in the r.get() call...

Answer 1

I'll document the answer in case anyone else finds this...

r = redis.StrictRedis()
with r.pipeline() as p:
    while 1:
        try:
            p.watch('FLOWCONTROL')  # --> immediate mode
            last_slot = float(p.get('FLOWCONTROL') or '0.0')
            p.multi()  # --> back to buffered mode
            my_slot = last_slot + SLOT_LENGTH
            p.set('FLOWCONTROL', max(my_slot, now))
            p.execute()  # raises WatchError if anyone changed TCTR-FLOWCONTROL
            break  # break out of while loop
        except WatchError:
            pass  # someone else got there before us, retry.

a little more complex than the original three lines...