I installed the ELK (version 5.6 elastic / logstash / kibana) stack on a CentOS 7 (using version 5.6) box using the rpm method, and have services enabled to run on startup. I verify it is running after reboot like so:
ps aux | grep logstash
logstash 744 4.6 2.2 3381144 363308 ? SNsl 09:06 0:48 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xms256m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash
I don't see anything in that process command regarding my pipeline configurations. In my settings file, I have path.config : /etc/logstash/conf.d/*.conf.
This is modified from the default path.config : /etc/logstash/conf.d/ which according to the documentation does the exact same thing (and I've tried both).
The 3 files I have are valid, as I can ingest data manually using:
/usr/share/logstash/bin/logstash -f '/etc/logstash/conf.d/{fileA,fileB,fileC}.conf' (or I can run them individually).
I want these ingestion pipelines to start when my server reboots, but I believe I have my configurations set correctly. I even re-ran the system-install script for good measure. Any ideas?
I ended up manually changing the systemd config for logstash by going to /etc/systemd/system/logstash.service and changing:
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
to:
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" "-f" "/etc/logstash/conf.d/{fileA,fileB,fileC}.conf"
After rebooting, I discovered the process was indeed loading these pipelines.