I have a Python program that uses aiohttp and ElementTree to fetch data from a website. The code below is a segment of a Discord chat bot hosted on a Raspberry Pi. The function works well most of the time, but after the bot has been on for a few days, the function begins to bog down, and always times out. Restarting the program doesn't fix the issue, only rebooting the Pi seems to solve the problem for a while. I know it's not a lot to go on, but is there an obvious issue with this segment of code that could case this, or does the problem lie somewhere else?
import lxml.etree as ET
import aiohttp, async_timeout
...
async with aiohttp.ClientSession() as session:
try:
with async_timeout.timeout(5):
async with session.get('https://example.com', params=params, headers=headers) as resp:
if resp.status == 200:
root = ET.fromstring(await resp.text(), ET.HTMLParser())
# Do stuff with root
else:
print("Error: {}".format(resp.response))
except Exception as e:
print("Timeout error {}".format(e))
Perhaps a memory leak somewhere that slowly uses up systems memory, once full everything becomes very slow as swap is used for memory allocation and timeouts occur.
However as Andrew says this can't be a problem with the python script or it would be fixed by restarting it.
Monitor system memory and go from there.