How to Dynamically log when iterating in a filepath and reading the contents of file How to create separate log file for each file?

Question

I have Created a maven java project, if given a folderpath, the code will iterate over the folder and pick file one by one and read all the contents of file (file type supported-txt,pdf,word,excel,csv,xml,logs) How to create separate log file for each file with same name as "filename_filetype.log" using log4j2 or any other lib will also work.

Answer 1

This is a recent example I was working with. The log4j2.xml I used is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>

    <Properties>
        <Property name="CONSOLE_PATTERN">%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n</Property>
        <Property name="LOG_PATTERN">%d %-5p method: [%t] %C{2} (%F:%L) - %m%n</Property>
    </Properties>

    <Appenders>
        <Routing name="RoutingAppender">
            <Routes pattern="$${marker:}">
                <Route>
                    <RollingFile name="RollingFile-${marker:}"
                                 fileName="./logs/application/${marker:}/app.log"
                                 filePattern="./logs/application/${marker:}/app-%d{MM-dd-yyyy}.log.gz"
                                 ignoreExceptions="false">
                        <PatternLayout pattern="${LOG_PATTERN}"/>
                        <TimeBasedTriggeringPolicy/>
                    </RollingFile>
                </Route>
            </Routes>
        </Routing>

        <Console name="StdOut" target="SYSTEM_OUT" follow="true">
            <PatternLayout pattern="${CONSOLE_PATTERN}"/>
        </Console>

        <Routing name="SequenceRoutingAppender">
            <Routes pattern="$${marker:}">
                <Route>
                    <RollingFile name="Sequencing-${marker:}"
                                 fileName="./logs/service/${marker:}/seq_gap.log"
                                 filePattern="./logs/service/${marker:}/seq_gap-%d{MM-dd-yyyy}.log.gz"
                                 ignoreExceptions="false">
                        <PatternLayout pattern="${LOG_PATTERN}"/>
                        <TimeBasedTriggeringPolicy/>
                    </RollingFile>
                </Route>
            </Routes>
        </Routing>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="RoutingAppender"/>
        </Root>
        <logger name="SequencingLogger" additivity="false" level="info">
            <AppenderRef ref="SequenceRoutingAppender"/>
        </logger>
    </Loggers>
</Configuration>

There are two RoutingAppenders I used, but its basically the same for both, I just want to log different things for the same Markers. It would take the marker values and insert it where the placeholders are ${marker:}. Inside my code, I have instantiated Markers as follows:

    Marker mainMarker = MarkerManager.getMarker("MAIN");
    Marker applicationMarker = MarkerManager.getMarker("APPLICATION");
    Marker sequenceMarker = MarkerManager.getMarker("SEQUENCE");

You can pass the name of your files and their file types as params to getMarker(). Then, when you log you pass the marker to the logger as well as whatever statement you want to log as follows:

LOGGER.warn(mainMarker, "unable to find config in working directory. Fall back to jar config");

I know that the answer I provide is quite brief and to the point without much explanation why, but here is an informative site that explains what is going on, and why some of the things are done the way they are. I used it quite frequently while trying to understand how RoutingAppender works since the sample provided by Apache, and other examples on SO were quite brief.

and quoting from the link

The Routing is kept under Appender and it’s a RoutingAppender.

The Routes element accepts a single attribute named “pattern”. The pattern is evaluated against all the registered Lookups and the result is used to select a Route. Each Route may be configured with a key. If the key matches the result of evaluating the pattern then that Route will be selected. If no key is specified on a Route then that Route is the default. Only one Route can be configured as the default.

The Routes element may contain a Script child element. If specified, the Script is run for each log event and returns the String Route key to use.

You must specify either the pattern attribute or the Script element, but not both.

Each Route must reference an Appender. If the Route contains a ref attribute then the Route will reference an Appender that was defined in the configuration. If the Route contains an Appender definition then an Appender will be created within the context of the RoutingAppender and will be reused each time a matching Appender name is referenced through a Route.

In our example the Routes pattern is $${ctx:module}. This pattern matches with a key called module in the context.

but in my case the Routes pattern in $${marker:}