Search code examples
graylog2graylog

Graylog ; How to limit extractors application to a subset of sources

Trying to use extractors within Graylog, I cannot find a way to limit the pattern matching to one source.

Basically I do a split&Index search to extract a field but I want this extractors to be used only for a subset of my sources. Documentation seems poor for this.

Any Idea ?

Thanks Loïc

Solution

  • Seems like this cannot be easily done. There're two options: pipelines and separate inputs. Obviously, if you capture different sources in different inputs then you prevent this problem.

    The second solution is to use Pipelines (as of v2). Here's the author confirming this:

    This is possible since Graylog 2.0 by using pipeline rules. http://docs.graylog.org/en/2.1/pages/pipelines.html

    p.s. I thought it might be possible to store and extract from the full message, but couldn't figure how to first cut the json prior to extraction. A sample message coming from a docker container might look like this (sending logs over syslog to Graylog):

    <30>1 2016-11-26T22:22:38.951321+01:00 www.example.com docker 19459 - -  {"name":"my-awesome-app","hostname":"docker24.example.com","pid":1,"level":30,"msg":"happily serving customers","time":"2016-11-26T21:22:38.950Z"}

    So the entire field is not a proper JSON and Graylog JSON extractor would fail. You have your source here - www.example.com, so it's possible to configure the extractor to only run when this matches, but then the question is how to parse only the JSON section..