Search code examples
logstashgrok

Logstash filters: When is a filter executed?


I searched the logstash docs but i could not find out how logstash executes the filters.

I will explain by example:

  1. Multiple config files, apache.conf nginx.conf logic.conf Both nginx and apache config files contain a filter that will trigger if their type is met and add a tag called "please_do_logic".

  2. logic.conf contains several grok filters that will extract the request part from the previous grokked log lines in nginx.conf and apache.conf

I have 2 questions:

  • How does logstash decide which config file will be executed first?
  • How can you ensure the logic.conf part will be executed after apache.conf and nginx.conf have been executed? I know you can put everything in a single file / filter and go on from there but that would create messy config files and this would be a last resort measure.

Thanks


Solution

  • Logstash can't operate on more than one .conf simultaneously, or create some sort of workflow of configuration files, its just not supported/implemented that way. The .conf file tells logstash what inputs to read, what filters to apply, and how to output the events. You'll have to put everything in one configuration file. Of course, you can create how many .conf files as you like, and run multiple instances of logstash, each with different configuration.