Search code examples
logstashkibanaelastic-stackfilebeat

Logstash: Pipeline worker error, the pipeline will be stopped OrgJrubyExceptions SystemCallError

This is my workflow filebeat -> logstash -> elasticsearch -> kibana

  • logstash 7.8.1
  • Ubuntu 18.04 64 bit

My logstash keeps getting shutdown unexpectedly after running successfully for sometime with below error i'm not able to identify the reason for this error to look for solutions.

i had to restart logstash to load the data into elasticsearch whenever it gets shutdown.

Any suggestions are appreciated

enter image description here

Solution

  • it has been a while since i posted this question. i have figured out a solution for this problem, click this link to view the solution i posted in github issues.

    Reposting same answer below

    i moved from logstash barebone software installation to docker image. As i expected, the docker container keeps running in the background without the issues that i have mentioned in my question. The container should be running in detached mode.

    Below is the command that is used to run the logstash in docker container mode

    docker run -d -p 9600:9600 -p 5044:5044 -v /home/elkuser/elkstack/softwares/logstash_file_docker/logstash.yml:/usr/share/logstash/config/logstash.yml -v /home/elkuser/elkstack/softwares/logstash_file_docker/pipeline-final.conf:/usr/share/logstash/pipeline/logstash.conf cc67e625d974

    Logstash docker image that is used docker pull docker.elastic.co/logstash/logstash:7.8.1

    References