Why is logstash merging tables?

Question

I've got two configuration files pointing each on a different table of the database but in elasticsearch, i get the two tables merged so, tablea and tableb items will all be copy as typea and tablea and tableb items will all be copy as typeb.

But i configured it to get tablea as typea and tableb as typeb.

Here's my configuration filea

input {

    jdbc {
        jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://db:3306/nombase"
        jdbc_user => "root"
        jdbc_password => "root"
        schedule => "* * * * *"
        statement => "SELECT * FROM `tablecandelete`"
    }

}


## Add your filters / logstash plugins configuration here


output {

    stdout { codec => json_lines }

    if [deleted] == 1 {
        elasticsearch {
            hosts => "elasticsearch:9200"
            index => "nombase"
            document_type => "tablecandelete"

            ## TO PREVENT DUPLICATE ITEMS
            document_id => "tablecandelete-%{id}"
            action => "delete"
        }
    } else {
        elasticsearch {
            hosts => "elasticsearch:9200"
            index => "nombase"
            document_type => "tablecandelete"

            ## TO PREVENT DUPLICATE ITEMS
            document_id => "tablecandelete-%{id}"
        }
    }

}

Here's my configuration fileb

input {

    jdbc {
        jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://db:3306/nombase"
        jdbc_user => "root"
        jdbc_password => "root"
        schedule => "* * * * *"
        statement => "SELECT * FROM `nomtable`"
    }

}


## Add your filters / logstash plugins configuration here


output {

    stdout { codec => json_lines }

    elasticsearch {
        hosts => "elasticsearch:9200"
        index => "nombase"
        document_type => "nomtable"

        ## TO PREVENT DUPLICATE ITEMS
        document_id => "nomtable-%{id}"
    }

}

Does anybody have an idea? Thanks for any response/help

Answer 1

While you do have two different configuration files, logstash doesn't treat them as independent -- you could have all of your inputs in one file, all of your outputs in another file.

To separate things out, you have to add type => something_unique on each of your inputs and then surround the remaining code in the config file with

if [type] == 'something_unique' { 
  // config here 
}