Making a logging.Handler with async emit

Question

I have a Python log handler that writes using asyncio (it's too much work to write to this particular service any other way). I also want to be able to log messages from background threads, since a few bits of code do that. So my code looks basically like this (minimal version):

class AsyncEmitLogHandler(logging.Handler):
    def __init__(self):
        self.loop = asyncio.get_running_loop()
        super().__init__()

    def emit(self, record):
        self.format(record)
        asyncio.run_coroutine_threadsafe(
            coro=self._async_emit(record.message),
            loop=self.loop,
        )

    async def _async_emit(message):
        await my_async_write_function(message)

Mostly it works fine but when processes exit I get a lot some warnings like this: "coroutine 'AsyncEmitLogHandler._async_emit' was never awaited"

Any suggestions on a cleaner way to do this? Or some way to catch shutdown and kill pending writes? Or just suppress the warnings?

Note: the full code is [here][1] [1]: https://github.com/lsst-ts/ts_salobj/blob/c0c6473f7ff7c71bd3c84e8e95b4ad7c28e67721/python/lsst/ts/salobj/sal_log_handler.py

Answer 1

What about this solution which will block closing until everything have finished emitting?

I'm using this where I use the QueueHandler and QueueListener as well to have the logging happen in a separate thread (this is also why I create a new event loop)

class AsyncEmitLogHandler(logging.Handler):
    def __init__(self):
        super().__init__()
        self.loop = asyncio.new_event_loop() 

    def emit(self, record):
        self.format(record)
        self.loop.create_task(self._async_emit(record.message))

    async def _async_emit(self, message):
        await my_async_write_function(message)

    def close(self) -> None:
        self.loop.run_until_complete(asyncio.gather(*asyncio.all_tasks(self.loop)))
        self.loop.close()