Search code examples
javaspringlog4jlog4j2

Understanding the Log4j configuration

I am currently working on tidying up the logging for one of our internal tools. I am new to Log4j & trying to best understand how to go about doing this.

Context

Our tool uses Log4j to log out information of the various processes the tool is executing. Currently, the log is quite bloated & is outputting too many log statements.

I've managed to cut down on some of the log statements by removing the logger.debug("..."); lines.

But it seems that some of the logging statements from the APIs/microservices we use are still getting logged. For example, we use some of Netflix's microservices & there are log statements from these microservices.

I understand the configuration files (log4j2.xml or log4j.properties) define how the logging is set up?

Here's the contents of the log4j2.xml file inside src/:

<?xml version="1.0" encoding="UTF-8"?>
<configuration status="OFF">
    <Appenders>
        <Console name="stdout" target="SYSTEM_OUT">..
            <PatternLayout pattern="."/>
        </Console>
        
        <RollingFile name="FILE" fileName="${sys:microservice.log}" filePattern="${sys:microservice.log}.%i">
            <PatternLayout>
                <Pattern>...</Pattern>
            </PatternLayout>
            <Policies>
                <OnStartupTriggeringPolicy />
                <SizeBasedTriggeringPolicy size="300 MB" />
            </Policies>
            <DefaultRolloverStrategy max="5" />
        </RollingFile>
    </Appenders>

    <Loggers>
        <Logger name="org.mongodb.driver" level="off" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Logger>
        <Logger name="com.ulisesbocchio" level="off" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Logger>
        <Logger name="org.springframework" level="warn" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Logger>
        <Logger name="org.springframework.jdbc.core" level="off" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Logger>
        <Logger name="org.springframework.jdbc.core.StatementCreatorUtils" level="off" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Logger>
        <Logger name="com.myOrgName" level="trace" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Logger>
        <Root level="info" additivity="false">
            <appender-ref ref="${sys:logger.out.type}"/>
        </Root>
    </Loggers>
</configuration>

At the same time, we have a log4j.properties file inside resources/:

####### ADP category definition.

# Category
log4j.rootLogger=TRACE,${logger.out.type}

log4j.logger.org.mybatis.example.BlogMapper=TRACE

# Console appender
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=...



log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
#log4j.appender.stdout.layout.ConversionPattern=%5p [%t] - [%d] %m%n
log4j.appender.stdout.layout.ConversionPattern=...

log4j.appender.FILE=org.apache.log4j.RollingFileAppender  
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout  
log4j.appender.FILE.layout.ConversionPattern=...
log4j.appender.FILE.File=${microservice.log}  
log4j.appender.FILE.MaxFileSize=100MB  
log4j.appender.FILE.MaxBackupIndex=5

My Questions

  1. There are logs coming from the Netflix microservice, how do I "switch it off" (for lack of a better word)?

  2. Based off a few tutorials & guides I read through, the Logger element inside Loggers should be corresponding to an Appender inside Appenders through <appender-ref ref="..."/>. But based on how the log4j2.xml file is written, most of the Loggers don't have a corresponding Appender? Is this correct? How does this work?

  3. What does ${sys:logger.out.type} in the <appender-ref ref="${sys:logger.out.type}"/> element mean?

  4. If I understand correctly, each Logger defines the details for how a package's log statements should appear? If a package does not have a Logger defined for it, will it get logged (by default) under the Root element?

  5. The configuration has a status of OFF. Does this mean all log statements are disabled? But log statements are getting printed in the console, so I'm confused

  6. I am getting log statements from a myBatis interface, I don't see an implementation for it & I understand Autowiring an interface that does not have an implementation will result in errors. I want to remove these log statements, how do I go about doing this?

  7. If I understand correctly, the log4j2.xml file inside src/ is what controls the logging configuration? The log4j.properties inside the build/ folder is simply a "by-product" or where Eclipse stores the binary files, etc, am I right?

I've been reading through various tutorials & guides & managed to make good progress with some other tasks: changing the format of logging statements & removing some logs

Solution

  • First of all, you seem to have log4j 1 and log4j 2 in your applications. I take it that each application would either use one or the other, not both in parallel (although it might be possible).

    Both logging libraries have a similar configuration file structure and search it in similar ways:

    Configuration of Log4j 2 can be accomplished in 1 of 4 ways:

    • Through a configuration file written in XML, JSON, YAML, or properties format.
    • Programmatically, by creating a ConfigurationFactory and Configuration implementation.
    • Programmatically, by calling the APIs exposed in the Configuration interface to add components to the default configuration.
    • Programmatically, by calling methods on the internal Logger class.

    That should explain why you have log4j.properties and log4j2.xml on the classpath. Why your project structure has one of them in the src folder while the other one just shows up in the build folder is something you have to figure out.

    To answer your questions:

    1. Either the Netflix microservice has it's separate set of logging configuration or it is embedded into something else. In one case you can simply deactivate all logging, in the other case figure out the category used for logging messages. Then modify your configuration to set that category to a high priority level, which means all lower priorities will be suppressed by log4j anyway.

    2. A logger is almost the same as a category. The application logs to 'Loggers', which can filter up to a level and forward the messages to Appenders. Unless you specify a logger's appender it will use the parent or root logger's appender.

    3. Check Property Substitution

    4. You understand correctly

    5. I never used the configuration status, but according to https://logging.apache.org/log4j/2.x/manual/configuration.html you can use it to better understand log4j behaviour (troubleshooting, debugging log4j issues)

    6. A category does not always have to match a class name, although this is a common pattern. At runtime an application can create loggers with arbitrary category names. If you want to remove it from the logs, just configure that logger with a high priority level, or use 'off'

    7. See my first paragraph. Log4j searches in several locations, I often use a resource on the classpath. Why your builds treat the log4j 1 configuration differently than log4j2 configuration is not possible to know from here. Dig into your project history for that.

    All in all you need to be aware of this page: https://logging.apache.org/log4j/2.x/manual/configuration.html