logstash nil import errors

Question

I'm getting some errors attempting to do a data import in logstash. I'm seeing it for every "geo" field that I have. Here are some of my config files

input {
    jdbc {
        jdbc_driver_library => "c:\binaries\driver\ojdbc6.jar"
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        jdbc_connection_string => "jdbc:oracle:thin:@random:1521/random"
        jdbc_user => "user"
        jdbc_password => "password"

        statement => "select a.*, myfunc() as geo from foo a"

        type => "sometype"
    }
}

filter{
    if [type] == "sometype" {
        mutate {
            rename => { "sometype_id" => "id" }
            remove_field => ["gdo_geometry"]
            add_field => [ "display", "%{id}" ]
        }

        # parses string to json
        json{
            source => "geo"
            target => "geometry"
        }
    }
}

output {
    if [type] == "sometype" {
        elasticsearch {
            hosts => ["myesbox:80"]
            document_id => "%{id}"
            index => "sjw"
        }
    }
}

Here is a second.

input {
    jdbc {
        jdbc_driver_library => "c:\binaries\driver\ojdbc6.jar"
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        jdbc_connection_string => "jdbc:oracle:thin:@random:1521/random"
        jdbc_user => "user"
        jdbc_password => "password"

        statement => "select a.*, myfunc() as geo from foo2 a"

        type => "sometype2"
    }
}

filter{
    if [type] == "sometype2" {
        mutate {
            rename => { "sometype2_id" => "id" }
            remove_field => ["gdo_geometry"]
            add_field => [ "display", "%{id}" ]
        }

        # parses string to json
        json{
            source => "geo"
            target => "geometry"
        }
    }
}

output {
    if [type] == "sometype2" {
        elasticsearch {
            hosts => ["myesbox:80"]
            document_id => "%{id}"
            index => "sjw"
        }
    }
}

And here is the error message (repeated once for each record in my database tables).

{:timestamp=>"2016-01-05T13:33:18.258000-0800", :message=>"Trouble parsing json", :source=>"geo", :raw=>nil, :exception=>java.lang.ClassCastException: org.jruby.RubyNil cannot be cast to org.jruby.RubyIO, :level=>:warn}

Now interestingly, the field DOES seem to import successfully. I can see the data populated as expected. But I don't know why this warning is being generated. I'm running the logstash as

logstash -f /my/logstash/dir

Also interesting to note is that if I modify the first config file given and changed the source json filter name to "geom" instead of "geo" -- this warning would no longer occur. It seems to only occur when I have multiple config files with the same field/json filter combinations. So if I then added a third config file and it had a "geo" field being parsed by the json filter -- the issue occurs again -- though I would still not see any warning messages for the first config file -- only the second and third.

Answer 1

The issue here actually turned out to be a bug with the 2.0 version of logstash. I'm not sure what exactly the problem was, but upgrading to 2.1 resolved the issue for me.