Search code examples
elasticsearchlogstashlogstash-configuration

Extracting xpath values in Logstash to conditionally create new fields


The structure of my xml file is attahced below . I want to extract the value of field created by xpath and run a if else condition on it. In a way that if that if value of station_name is equal to "Finch" then create a new field and store in it.Can anyone suggest me how can I achieve this

A codified version of what I am trying to achieve

if "Finch" in [station_name] {
xpath => ["/station/name/text()","Ny_station"]
}
else {
xpath => ["/station/name/text()","nonNy_station"]
}

Here is my config file

input
{
file
{
path => "C:\Users\186181152\Downloads\stations3.xml"
start_position => "beginning"
sincedb_path => "/dev/null"
exclude => "*.gz"
type => "xml"
codec => multiline {
     pattern => "<stations>" 
     negate => "true"
     what => "previous"
}
}
}
filter
{
xml
{
source => "message"
store_xml => false
target => "stations"
xpath => [
"/stations/station/id/text()", "station_id",
"/stations/station/name/text()", "station_name"
]
}

}

output
{
elasticsearch
{
codec => json
hosts => "localhost"
index => "xmlns24"
}
stdout
{
codec => rubydebug
}

}

enter image description here


Solution

  • You can't the if else loop in the xml filter. But you can have two xml filter and in the if else loop, with the only difference being the variable created by the xpath.

    Here it checks the string Finch as a regex pattern against the message value to decide which xml filter to use.

    if [message] =~ "Finch" {
        xml
        {
            source => "message"
            store_xml => false
            target => "stations"
            xpath => [
                "/stations/station/id/text()", "station_id",
                "/stations/station/name/text()", "Ny_station"
            ]
        }
    } else {
        {
            source => "message"
            store_xml => false
            target => "stations"
            xpath => [
                "/stations/station/id/text()", "station_id",
                "/stations/station/name/text()", "nonNy_station"
            ]
        }
    }