Logstash output csv specific rows

Question

I have a question that has been bugging me in the paste few day I and I have tried different approaches but nothing seems to work.

I am trying to write some syslog output on my local disk in a css file. So following the documentation I set this output:

output {
  elasticsearch {
     hosts => ["localhost:9200"]
         index => "logstash_index"
 }
  stdout {
    codec => rubydebug
 }
  csv {
    path => "path-to-file\syslogs-%{+yyyy.MM.dd}.csv"
    csv_options => {
        "write_headers" => true
    "headers" => ["timestamp", "message", "count"]
}
    fields => ["@timestamp", "message", "count"]
 }
}

This works just fine, even if for each syslog entry it generate the headers, but is not a big deal for now. What I am really trying to achieve here, is that I don't want all the syslog to be written to css, but only specific row.

here an example to make it clear.

assuming I have this data structure

timestamp.   message     id.      count
13.05.       hello 1.    01.        2
10.05        hello 2.    02.     
13.05.       hello 3.    03.  

in my local css file I would like to save only the rows that contain a count field, and if the count is empty, to be ignored and not saved.

can anyone help me please to understand what would be the best approach to solve this issue? because for now, the output is saving all the longs, and I would like to save space on my hard disk if its possible

Thank you so much guys

Answer 1

You can use a conditional in the output section...

if [count] {
    csv {
        path => "path-to-file\syslogs-%{+yyyy.MM.dd}.csv"
        csv_options => {
            "write_headers" => true
        "headers" => ["timestamp", "message", "count"]
        }
        fields => ["@timestamp", "message", "count"]
    }
}