Search code examples
javalog4jlogstashlogstash-grok

log4j grok with pipes fail


i need to format a grok input for the next log format

[#|2017-10-12T07:40:16.232-0700|INFO|sun-appserver2.1|RugDaemonCMLog|_ThreadID=17;_ThreadName=Thread-25;|Wake Up!!#]

[#|2017-10-12T07:40:16.241-0700|INFO|sun-appserver2.1|RugDaemonCMLog|_ThreadID=17;_ThreadName=Thread-25;|--Don't have a work in batch|#]

[#|2017-10-12T07:40:16.241-0700|INFO|sun-appserver2.1|RugDaemonCMLog|_ThreadID=17;_ThreadName=Thread-25;|go to sleep!|#]

[#|2017-10-12T07:40:16.567-0700|INFO|sun-appserver2.1|RugProject|_ThreadID=16;_ThreadName=Thread-23;| sleep---|#]

[#|2017-10-12T07:40:16.568-0700|INFO|sun-appserver2.1|RugProject|_ThreadID=16;_ThreadName=Thread-23;|Sending Mail|#]

[#|2017-10-12T07:40:16.569-0700|INFO|sun-appserver2.1|RugProject|_ThreadID=16;_ThreadName=Thread-23;|--lookup--|#]

i have the following sintax in my log4j file

log4j.appender.file.datePattern='.'yyyy-MM-dd_HH_mm
log4j.appender.file.MaxFileSize=10MB
log4j.appender.file.MaxBackupIndex=100
log4j.appender.file.encoding=UTF-8
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{dd-MM-yyyy HH:mm:ss} %-5p %c{1}:%L - %m%n

in the debugger online i'm trying this format

%{TIMESTAMP_ISO8601:logdate}\|%{LOGLEVEL:loglevel}|%{WORD:caller}\|%{NONNEGINT:line} - %{GREEDYDATA:message}$

that format parse the date and the log level info, but not the caller and the thread information


Solution

  • Use this:

    %{TIMESTAMP_ISO8601:logdate}\|%{LOGLEVEL:loglevel}\|%{DATA:caller}\|%{DATA:line}\|_ThreadID=%{DATA:threadid};_ThreadName=%{DATA:threadname}\|%{GREEDYDATA:message}\|%{GREEDYDATA:fin}$