Search code examples
python-asynciopython-3.9

Asyncio with memory leak (Python)


I have an issue with my (dockerized) application consuming an ever increasing amount of memory until the Linux kernel kills is. In the container's docker logs I only got an ominous Killed message without further context - only after checking the kernel logs (cat /var/log/kern.log) and docker stats did I realize what was happening - memory usage was going up by ~10MB per second.

The application is an asyncronous grpc-server with several concurrent tasks (using return ensure_future(self._continuous_function) to keep the function's task up and running, async so the server endpoints are not blocked).


Solution

  • I figured out that ensure_future() caused my memory leak issue - or rather the return I used with it. Apparently the return meant that a reference to the original task was being kept in memory (instead of being garbage collected), and the function only had a very short wait time associated with it (1ms) - so memory kept building up fast.

    So after removing the leak my app now consumes about 60MB of very constant memory. I hope this helps somebody else out, I couldn't find a reference to this exact issue which is why I'm sharing it here.