Search code examples
logbackapache-stormstream-processing

How to configure logback-based logging to handle log masking with Apache Storm?


I am trying to configure logback based log masking for Apache Storm topologies. When I try to replace logback.xml file inside Apache Storm log4j2- directory and update worker.xml and cluster.xml file, Apache Storm nimbus and supervisors are unable to understand logback based keywords.

Error:

    2022-10-02 16:31:51,671 Log4j2-TF-1-ConfiguratonFileWatcher-2 ERROR Unable to locate appender "A1" for logger config "root"
    2022-10-02 16:32:51,681 Log4j2-TF-7-ConfiguratonFileWatcher-4 ERROR Error processing element appender ([configuration: null]): CLASS_NOT

Sample cluster.xml file:

    <configuration monitorInterval="60" shutdownHook="disable">
    <properties>
        <property name="pattern">%msg%n</property>
    </properties>
    
    <import class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"/>
    <import class="ch.qos.logback.core.FileAppender"/>
    
    <FileAppender name="A1">
            <file>logfilename.log</file>
            <encoder>
                <pattern>${pattern}</pattern>
            </encoder>
    </FileAppender>
    
    <loggers>
        <root level="info"> <!-- We log everything -->
            <appender-ref ref="A1"/>
        </root>
    </loggers>
</configuration>

Solution

  • To my best knowledge, Apache Storm uses naturally log4j2, as also your logfile indicates. However, when I used log4j in Storm, I did not need to import any further classes. You also do not seem to use these logback-classes in the rest of your xml-file. So have you tried to simply remove those?