Search code examples
rubylogstash

How to use a gem in logstash pipeline ruby filter


I have a ruby file with dependency modules and this file is being called in a logstash pipeline. ie ...

filter{

  ruby {
    path  => "/path/to/script.rb"
  }

}

I get an error that they can't be found when the script is called through the logstash pipeline in the pod where its running.

How or where do I install these? Can I run "gem install my-gem" somewhere in the pipeline or how would you go about completing this?


Solution

  • There is no official way to install a gem (yet at least). However there might be a few options:

    1. You can add a Gem to /usr/share/logstash/Gemfile and then use them gem in your script as usual. Apparently this can be bit flaky since an update to logstash might need a re-install of the gem.

    2. Write a custom plugin, but that can be an overkill for what you might want.

    3. As the author of the ticket linked below said, you can: add you Gem to logstash's Gemfile at /usr/share/logstash/Gemfile, force a gem update by running /usr/share/logstash/bin/logstash-plugin update logstash-filter-grok and then use it in your Ruby filter. Ref to the ticket below for more details.

    NOTE: This answer uses information referenced in this ticket here:

    https://github.com/elastic/logstash/issues/7016

    Hope that helps.