Concatenate a word to an email using pre-defined grok filter

Question

first of all thank you for reading my question. i have an email address in a log in following format,

Apr 24 19:38:51 ip-10-0-1-204 sendmail[9489]: w3OJco1s009487: sendid:name@test.co.uk, delay=00:00:01, xdelay=00:00:01, mailer=smtp, pri=120318, relay=webmx.bglen.net. [10.0.3.231], dsn=2.0.0, stat=Sent (Ok: queued as E2DEF60724), w3OJco1s009487: to=<username@domain.us>, delay=00:00:01, xdelay=00:00:01, mailer=smtp, pri=120318, relay=webmx.[redacted].net. [10.0.3.231], dsn=2.0.0, stat=Sent (Ok: queued as E2DEF60724)

and i need to extract the email along with the word sendid

output should look like this,

{
  "DATA": [
    [
      "sendid:name@test.co.uk"
    ]
  ]
}

i have tried following but it only extracts email i tested it here, http://grokdebug.herokuapp.com/ ,

sendid:%{DATA},

How can i concatenate the word sendid: to the email without creating a new field or defining a new regex? can someone please help?

i have also tried this but it doesn't work,

sendid:%{"sendid:"} %{DATA},

Answer 1

Your sendid:%{DATA}, won't work because anything that you provide outside grok pattern are matched as surroundings, in your case everything between sendid: and , will be matched, and it will give you,

{
  "DATA": [
    [
      "name@test.co.uk"
    ]
  ]
}

You need to create a custom pattern and combine it with pre-defined pattern for your solution, since you cannot use any pre-defined pattern entirely.

Logstash allows you to create custom patterns using Oniguruma regex library for such situations. The syntax is,

(?<field_name>the pattern here)

in your case it will be,

\b(?<data>sendid:%{EMAILADDRESS})\b

OUTPUT:

{
  "data": [
    [
      "sendid:name@test.co.uk"
    ]
  ],
  "EMAILADDRESS": [
    [
      "name@test.co.uk"
    ]
  ],
  "EMAILLOCALPART": [
    [
      "name"
    ]
  ],
  "HOSTNAME": [
    [
      "test.co.uk"
    ]
  ]
}