Parsing Apache 2 Error Log with Grok Debugger

Question

I'm trying to set up a pattern in the grok debugger at http://grokdebug.herokuapp.com/

Input:

[Sat Aug 01 21:54:54.048805 2015] [:error] [pid 4384:tid 140066215139072] [client 192.168.1.1:62028] PHP Notice:  Undefined index: foo in /home/koan/websightdesigns/websightdesigns.com/ierr.php on line 3

Pattern:

\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\]\s\[:%{LOGLEVEL:loglevel}\]

What I have so far works, it outputs:

{
  "timestamp": [
    [
      "Sat Aug 01 21:54:54.048805 2015"
    ]
  ],
  "day": [
    [
      "Sat"
    ]
  ],
  "month": [
    [
      "Aug"
    ]
  ],
  "MONTHDAY": [
    [
      "01"
    ]
  ],
  "TIME": [
    [
      "21:54:54.048805"
    ]
  ],
  "HOUR": [
    [
      "21"
    ]
  ],
  "MINUTE": [
    [
      "54"
    ]
  ],
  "SECOND": [
    [
      "54.048805"
    ]
  ],
  "YEAR": [
    [
      "2015"
    ]
  ],
  "loglevel": [
    [
      "error"
    ]
  ]
}

So far, so good. However, the problem I am having is that if I try to add on to my pattern to get the next section, the [pid 4384:tid 140066215139072] section, I get a compile error no matter I try.

I've tried:

\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\]\s\[:%{LOGLEVEL:loglevel}\]\s\[%{PID:pid}\]

And:

\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\]\s\[:%{LOGLEVEL:loglevel}\]\s\[%{PID:pid}:%{TID:tid}\]

As well as other patterns, but nothing seems to be working. Anyone out there know what I might be doing wrong in trying to get [pid 4384:tid 140066215139072] into variables?

Answer 1

After working on this some more, the following pattern is now working for me:

\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[.*:%{LOGLEVEL:loglevel}\] \[pid %{NUMBER:pid}:tid %{NUMBER:tid}\] \[client %{IP:clientip}:.*\] %{GREEDYDATA:errormsg}