Search code examples
mongodbelasticsearchkibanalogstash-configuration

Index not creating in kibana for mongodb + logstash +elasticsearch

I'm trying to visualize the mongodb data in kibana using logstash configuration.Below is my configuration.I'm getting some outputs in terminal and it is looping forever. I couldn't see any index created by the name mentioned in the config file and if the index was generated also don't have any data on it. Saying no results to match in the discover tab.How to make the configuration to visualize the data in kibana?

 input {
            mongodb {
            uri => "mongodb+srv:@@@@@@@@@@@@@@@?retryWrites=true&w=majority"
            placeholder_db_dir => "C:/logstash-mongodb"
            placeholder_db_name => "logstash1_sqlite.db"
            collection => "logs"
            batch_size => 1
            
            
            }
    }
    filter {
    
    }
    output {
            stdout {
                    codec => rubydebug
            }
            elasticsearch {
                    action => "index"
                    index => "ayesha_logs"
                    hosts => ["localhost:9200"]
            }
    }

enter image description here

http://localhost:9200/ayesha_logs/_search?pretty

enter image description here

Terminal logs: D, [2020-10-01T08:11:45.717000 #2372] DEBUG -- : MONGODB | range-api-test-cluster-shard-00-02.icqif.azure.mongodb.net:27017 req:259 conn:1:1 sconn:231839 | coexistence-poc.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "nameOnly"=>true, "$db"=>"coexistence-poc", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x32598cb2 @increment=1, @seconds=1601532700>, "signature"=>{"hash"=><BSON::Binary:0x2622 type=generic data=0xfaf25a8d85... D, [2020-10-01T08:11:45.755000 #2372] DEBUG -- : MONGODB | range-api-test-cluster-shard-00-02.icqif.azure.mongodb.net:27017 req:259 | coexistence-poc.listCollections | SUCCEEDED | 0.038s D, [2020-10-01T08:11:50.801000 #2372] DEBUG -- : MONGODB | range-api-test-cluster-shard-00-02.icqif.azure.mongodb.net:27017 req:260 conn:1:1 sconn:231839 | coexistence-poc.find | STARTED | {"find"=>"coexistence-pinfobackfill-logs", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5f71f009b6b9115861d379d8')}}, "limit"=>50, "$db"=>"coexistence-poc", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x32598cb2 @increment=1, @seconds=1601532700>, ... D, [2020-10-01T08:11:50.843000 #2372] DEBUG -- : MONGODB | range-api-test-cluster-shard-00-02.icqif.azure.mongodb.net:27017 req:260 | coexistence-poc.find | SUCCEEDED | 0.042s D, [2020-10-01T08:11:50.859000 #2372] DEBUG -- : MONGODB | range-api-test-cluster-shard-00-02.icqif.azure.mongodb.net:27017 req:261 conn:1:1 sconn:231839 | coexistence-poc.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "nameOnly"=>true, "$db"=>"coexistence-poc", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x32598cb2 @increment=1, @seconds=1601532700>, "signature"=>{"hash"=><BSON::Binary:0x2622 type=generic data=0xfaf25a8d85... D, [2020-10-01T08:11:50.906000 #2372] DEBUG -- : MONGODB | range-api-test-cluster-shard-00-02.icqif.azure.mongodb.net:27017 req:261 | coexistence-poc.listCollections | SUCCEEDED | 0.047s

enter image description here

Solution

  • Did you create your Kibana's index pattern ? If not, just go to Menu > stack managment > Kibana > Index pattern

    click on enter image description here

    And follow the steps. enter image description here

    You will then be able to use you index in Discover or visualization tabs.