Search code examples
regexfilebeat

Filebeat regex - whitespace before digits


I have the following config in my filebeat.yml :

- type: log
  close_renamed: true
  paths:
    - /logs/example.log
  multiline:
    pattern: '^[A-Za-z]{3} [A-Za-z]{3} [0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} [0-9]{4}'
    negate: true
    match: after
    timeout: 3s
  fields_under_root: true
  fields:
    type: oracle
    sourcetype: oracle
  tags: ["oracle"]

Example.log (truncated) :

...
Thu Oct  1 23:01:00 2020 +00:00
LENGTH : '275'
ACTION :[7] 'CONNECT'
DATABASE USER:[1] '/'
PRIVILEGE :[6] 'SYSDBA'
CLIENT USER:[9] 'test_user'
CLIENT TERMINAL:[5] 'pts/0'
STATUS:[1] '0'
DBID:[10] '1762369616'
SESSIONID:[10] '4294967295'
USERHOST:[21] 'testdevserver'
CLIENT ADDRESS:[0] ''
ACTION NUMBER:[3] '100'

Thu Oct  1 23:01:00 2020 +00:00
LENGTH : '296'
ACTION :[29] 'SELECT STATUS FROM V$INSTANCE'
DATABASE USER:[1] '/'
PRIVILEGE :[6] 'SYSDBA'
CLIENT USER:[9] 'test_user'
CLIENT TERMINAL:[5] 'pts/0'
STATUS:[1] '0'
DBID:[10] '1762369616'
SESSIONID:[10] '4294967295'
USERHOST:[21] 'testdevserver'
CLIENT ADDRESS:[0] ''
ACTION NUMBER:[1] '3'

I noticed that this pattern '^[A-Za-z]{3} [A-Za-z]{3} [0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} [0-9]{4}' doesnt work for the example above Thu Oct 1 23:01:00 2020 +00:00 because there is a whitespace after Oct and before 1. How do I remove this whitespace so the pattern would match accordingly?

Thanks! J


Solution

  • If there are multiple spaces instead of a single space, you can use + to match 1 or more.

    Currently that would not get the desired match, as the day starts with a single digit 1. You can update the day part to match 1 or 2 digits using \d{1,2}

    ^[A-Za-z]{3} [A-Za-z]{3} +[0-9]{1,2} [0-9]{2}:[0-9]{2}:[0-9]{2} [0-9]{4}
    

    Regex demo

    You might for example make the pattern a bit more precise for the time and year part. You could extend it to also make the days and months an exact match.

    ^[A-Za-z]{3} +[A-Za-z]{3} +(?:[1-9]|[12]\d|3[01]) +(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d +(?:19|20)\d{2}\b
    

    Regex demo