I want to record STORM log by log4j and send log to kafka. the result is kafka STORM can record log by itself log4j but not send log to kafka

Question

I'm using ubuntu14.6,kafka_2.11-2.20,zookeeper-3.4.14,storm-1.2.3.when i use LocalCluster pattern can send log to kafka, i can get the log by kafka consumer. but i tend to submit topology by cluster pattern , it can't get log by kafka consumer. this is my log4j configuration

<configuration status="INFO">
    <appenders>
        <Kafka name="kafkaLog" topic="connect-test" ignoreExceptions="false">
        <PatternLayout pattern="%m%n" />
                <Property name="bootstrap.servers">192.168.23.165:9092</Property>
                <Property name="max.block.ms">2000</Property>
        </Kafka>

    </appenders>

    <loggers>
        <logger level = "info" name= "BasicDemo" additivity="false">>
            <appender-ref ref="kafkaLog"/>
        </logger>
        <root level="info">
            <appender-ref ref="kafkaLog"/>
        </root>
        <logger name="org.apache.kafka" level="INFO"/>
    </loggers>
</configuration>

there is a question log4j can send distribute system log to kafka ?

Answer 1

I don't think this question really has anything to do with Storm. I don't think the issue is with log4j either. You probably need to check that your distributed worker machines can actually connect to Kafka.

Also keep in mind that you will want to put your log4j config in storm/log4j2/worker.xml.