Search code examples
logstashkibanaelastic-stacklogstash-grokfilebeat

Java Strace gork pattern in logstash filter


i have following error log . I need to right the gork pattern for it . My pattern is only able to read first line . could anyone help me to read the other line and store the line in extraline field . Here the expression which i am currently on

\[%{NOTSPACE:thread}\] \[%{LOGLEVEL:loglevel}\] (?<timestamp>%{TIMESTAMP_ISO8601}) (?<logger>[A-Za-z0-9$_.]+):%{NOTSPACE:method}\(\):%{NONNEGINT:line} - %{GREEDYDATA:message}$

Log which needs to be parsed

[pool-4-thread-1] [ERROR] 2019-06-19 12:56:14,827 com.chaipoint.boxc.dao.OrderDispenseDataDao:updateOrderStatusOnDispenseScheduler():218 - Error while updating status in scheduler
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:627)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:906)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:930)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:313)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:318)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao.updateOrderStatusOnDispenseScheduler(OrderDispenseDataDao.java:216)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao$$FastClassBySpringCGLIB$$e49056b7.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

Output expected is


method : updateOrderStatusOnDispenseScheduler
timestamp : 2019-06-19·12:56:14,827
logger : com.chaipoint.boxc.dao.OrderDispenseDataDao
line : 218
thread : pool-4-thread-1
message : Error·while·updating·status·in·scheduler
loglevel : ERROR
extra line : org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:627)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:906)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:930)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:313)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:318)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao.updateOrderStatusOnDispenseScheduler(OrderDispenseDataDao.java:216)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao$$FastClassBySpringCGLIB$$e49056b7.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)




Solution

  • My pattern is only able to read first line

    To read multiple lines use: https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html

    For example, Java stack traces are multiline and usually have the message starting at the far-left, with each subsequent line indented. Do this:

    input {
      stdin {
        codec => multiline {
          pattern => "^\s"
          what => "previous"
        }
      }
    }