Search code examples
fluentd

Can I selectively disable individual fluentd plugins without removing them from the configuration file?


I have a docker-compose setup with containers logging into fluentd. To support different demo environments, I have events being output to multiple destinations (ElasticSearch, Splunk, Syslog, etc.)

I would like to maintain a single configuration file, but disable output plugins that are not needed. If I have 4 potential output destinations, I would have to maintain 10 different configuration files to support all the different possible combinations.

I know that plugins can use environment variables for configuration parameters, which would be ideal. However, I don't see that there is a common 'enabled' or 'disable' parameter in the underlying plugin architecture.

Is there any way to disable a plugin externally? Or will I have to dynamically build my configuration file from an external script?


Solution

  • I ended up doing this with environment variables by specifying the plugin type externally:

    <label @SPLUNK>
      <match docker.**>
        @type "#{ENV['FLUENTD_SPLUNK_PLUGIN']}"
        ...
    

    Externally, I either set FLUENTD_SPLUNK_PLUGIN=splunk_hec or FLUENTD_SPLUNK_PLUGIN=null based on whether the output should be enabled. So far the only downside to this is a bunch of warnings from the 'null' output plugin for the parameters that are defined in the config block.