Search code examples
pythonalgorithmdecoratorpython-decoratorsrate-limiting

Rate-limiting python decorator


I found this rate-limiting python decorator based on redis classes. How can I write a similar decorator that uses only what's available in the standard library that can be used as follows?

def ratelimit(limit, every):
    # 🐍 python magic 🐍

@ratelimit(limit=1, every=2)
def printlimited(x):
    print x

# print one number every two seconds
for x in range(10):
    printlimited(x)

There are other answers on stackoverflow but they do not allow to specify the denominator.


Solution

  • You can use a threading.Semaphore to count and block the requests that are exceeding the limit, in combination with threading.Timer to schedule a function that releases the semaphore.

    from threading import Semaphore, Timer
    from functools import wraps
    
    def ratelimit(limit, every):
        def limitdecorator(fn):
            semaphore = Semaphore(limit)
            @wraps(fn)
            def wrapper(*args, **kwargs):
                semaphore.acquire()
                try:
                    return fn(*args, **kwargs)
                finally:                    # don't catch but ensure semaphore release
                    timer = Timer(every, semaphore.release)
                    timer.setDaemon(True)   # allows the timer to be canceled on exit
                    timer.start()
            return wrapper
        return limitdecorator
    

    I extended this idea and published a library on PyPI named limit.