Search code examples
pythontensorflowloggingpython-logging

Tensorflow causes logging messages to double

So I was playing around with Google's Tensorflow library they published yesterday and encountered an annoying bug that keeps biting me.

What I did was setup the python logging functions as I usually do, and the result was that, if I import the tensorflow library, all messages in the console started doubling. Interestingly, this does not happen if you just use the logging.warn/info/..() function.

An example of a code that does not double the messages:

import tensorflow as tf
import logging

logging.warn('test')

An example of a code that does double all messages:

import tensorflow as tf
import logging

logger = logging.getLogger('TEST')
ch = logging.StreamHandler()
logger.addHandler(ch)

logger.warn('test')

Now, I'm a simple man. I like the functionality of logging, so I use it. The setup with the logger object and the adding of a StreamHandler is something I picked up looking at how other people did this, but it looks like it fits with how the thing was meant to be used. However, I do not have in-depth knowledge of the logging library, as it always just kind of worked.

So, any help explaining why the doubling of the messages occurs will be most helpful.

I am using Ubuntu 14.04.3 LTS with Python 2.7.6, but the error happens in all Python 2.7 versions I tried.

Solution

  • I get this output:

    test
WARNING:TEST:test

    Tensorflow is also using the logging framework and has set up its own handlers, so when you log, by default, it propagates up to the parent logging handlers inside tensorflow. You can change this behavior by setting:

    logger.propagate = False

    See also duplicate output in simple python logging configuration

    Followup: This was an unintended side-effect of the way tensorflow was using the logging package. I've changed it at HEAD to scope its internal loggers under the name "tensorflow" to avoid this pollution. Should be in the github head within a day or so. In the meantime, the logger.propagate solution will work and won't break once that fix is in, so you should be safe to go. Thanks again for spotting this!

    Followup-Followup: Starting with TensorFlow 1.14 exposes the logger directly:

    import tensorflow as tf

logger = tf.get_logger()