Search code examples
pythonloggingtwisted

how to use the twistd log system for logging my data?


I am interested in the main features of the twistd logging system, and I'm using it to log some data i need, much more that logging the real state of the twisted application. BTW it is very noisy, i've readed this Twisted: disable logging of Twisted-framework classes but i am not sure to get the point. Also using the .noise will not fit my need.

I would like to know if it is possible and easy to separate clearly in two logging systems having on one hand the log necessary for the twisted matrix application and on the other only the log that will contain my important data?

(so that i can have the time features, the rotating and so on on my own data, as i i've already spent some efforts with the kindly help of many guys from here to adapt the process of twisted logging to my needs).

can someone give me some hints on how to achieve this?

or maybe your main advice will be that i should definitively open a file, print my time format and data line inside it. And implements my own rotating on this file rather that turning away/hijack the twisted logging system to my needs?

i have also think of using a log.msg(mydata, system = "myownflag") and then using a grep myownflag | my log > only-my-data but there can be better ideas...

(i'm new to twisted, and learning it the wrong way, from the end and deeping too fast on my needs, rather that on the library basis, so i miss lot of thing. Please apologize me for that.)

best regards.


Solution

  • Here is a log observer wrapper that filters out events from the wrong system:

    from functools import wraps
    
    def makeObserver(system, originalObserver):
        @wraps(originalObserver)
        def observe(event):
            if event.get("system", None) == system:
                originalObserver(event)
    

    You can use this by wrapping any existing observer and adding it to the logging system:

    from twisted.python.log import FileLogObserver, addObserver
    
    fileObs = FileLogObserver(file("myownflag.log", "at"))
    addObserver(makeObserver("myownflag", fileObs.emit)