how to remove value from list safely in asyncio

There is a global list to store data. Different async functions maybe add or remove value from it.

Example:

a = [] # List[Connection]

async def foo():
    for v in a:
        await v.send('msg')
       

async def bar():
    await SomeAsyncFunc()
    a.pop(0)

Both foo and bar will give up the control to let other coroutines run, so in foo, it is not safe to remove value from the list.

Solution

The following example shows how to use the lock for this:

Create a connection manager:

import asyncio

class ConnectionsManager:
    def __init__(self, timeout=5):
        self.timeout = timeout
        self._lock = asyncio.Lock()
        self._connections = []
    
    async def __aenter__(self):
        await asyncio.wait_for(self._lock.acquire(), timeout=self.timeout)
        return self._connections

    async def __aexit__(self, *exc):
        self._lock.release()

The timeout is a security measure to break bugs with circular waits.

The manager can be used as follows:

async def foo():
    for _ in range(10):
        async with cm as connections:
            # do stuff with connection
            await asyncio.sleep(0.25)
            connections.append('foo')
        
async def bar():
    for _ in range(5):
        async with cm as connections:
            # do stuff with connection
            await asyncio.sleep(0.5)
            if len(connections) > 1:
                connections.pop()
            else:
                connections.append('bar')

cm = ConnectionsManager()
t1 = asyncio.create_task(foo())
t2 = asyncio.create_task(bar())
await t1
await t2
async with cm as connections:
    print(connections)

Note, that you could also be more explicit with connections here:

async def foo(cm):
    ...
async def bar(cm):
    ...

Just to make a comment why being explicit is so beneficial in contrast to globals. At some point you may need to write unit tests for your code, where you will need to specify all inputs to your functions/methods. Forgetting conditions on implicit inputs to your function (used globals) can easily result in untested states. For example your bar coroutine expects an element in the list a and will break if it is empty. Most of the time it might do the right thing, but one day in production...