Search code examples
logstash

Logstash get only message content


I would like to read a file line-by-line and put only the rows into a queue. The process works but in the queue there are a lot of other information such as version and so one.

Do you have any idea?

Here is my config file:

input {
   file {
     codec => plain
     path => "/apps/*.txt"
  }
}
filter{

}
output {
    stomp {
        host => "localhost"
        port => "61613"
        destination => "stomp-test"
        debug => "true"
        headers => {
                "amq-msg-type" => "text"
                "my_custom_header" => "kutya"
        }
    }
}

The example.txt contains: This is my test row!

The MQ body contains: {"message":"This is my test row!","@version":"1","@timestamp":"2017-01-21T13:42:21.084Z","path":"/apps/1.txt","host":"ol7"}

My expected result should be: This is my test row!

Thanks in advance.


Solution

  • The stomp output, as it currently stands, will only write JSON to the destination queue as can be seen in the source code of that plugin:

    t.send(event.sprintf(@destination), event.to_json, headers)
                                              ^
                                              |
                                         writes JSON
    

    The best you can do is to remove all other fields using a mutate/remove_field filter, like this:

    filter {
      mutate {
        remove_field => ["@timestamp", "@version", "path", "host"]
      }
    }
    

    Then you'll get only this in your destination queue:

    {"message":"This is my test row!"}