Logstash conditional output using environment variable not working

Question

I'm using Logstash to send log data to Elasticsearch (of course), but some of my end users also want the data sent to a secondary csv file so they can do their own processing. I am trying to use an environment variable to determine if we need to output to a secondary file and if so, where that file should live.

My Logstash looks like this:

input {
    . . .
}
filter {
    . . .
}
output {
    elasticsearch {
        . . .
    }
    if "${SECONDARY_OUTPUT_FILE:noval}" != "noval" {
        csv {   
            fields => . . .
            path => "${ SECONDARY_OUTPUT_FILE:noval}"
        }
    }
}

When SECONDARY_OUTPUT_FILE has a value, it works fine. When it does not, Logstash writes csv output to a file named "noval". My conclusion is that the if statement is not working correctly with the environment variable.

I'm using Logstash version 2.3.2 on a Windows 7 machine.

Any suggestions or insights would be appreciated.

Answer 1

Actually that is a very good question, there is still an ongoing enhancement opened on this topic on github, as mentioned by IrlJidel on the issue, a workaround to the issue would be:

mutate {
  add_field => { "[@metadata][SECONDARY_OUTPUT_FILE]" => "${SECONDARY_OUTPUT_FILE:noval}" }
}

if [@metadata][SECONDARY_OUTPUT_FILE] != "noval" {
      csv {   
        fields => . . .
        path => "${SECONDARY_OUTPUT_FILE}" 
      }
}