Index pattern is not showing up in Kibana management

Question

Using ELK Stack ver 7.3.2 Linux RedHat 6.9 (Santiago) Kafka Client 2.1
Using kafka-input-plugin with Logstash.
Kibana is up and running.
Elastic search is configured to run with Kibana.
Kibana is running on the same VM as Elastic search.
Hearbeat is running.
Filebeat is running.
Logstash input is a Kafka topic and is configured to output to Elastic.

I cannot see an index that supposed to be generated by Elastic and put on Kibana.

Here is the simple config for Logstash

input {
  kafka {
    bootstrap_servers => "nydvalkafvm01.tradeware.com:9092"
    topics => ["cat.dev1.service.NJPW7DT004.SM1"]
  }
}

output {
  elasticsearch {
    hosts => ["http://nyuatalcasvm01:9200"]
    index => "test_boris"
    workers => 1
    #user => "elastic"
    #password => "changeme"
  }
}

Here is the console output

[2019-09-20T22:02:00,444][INFO ][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] [Consumer clientId=logstash-0, groupId=logstash] Revoking previously assigned partitions []
[2019-09-20T22:02:00,445][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] [Consumer clientId=logstash-0, groupId=logstash] (Re-)joining group
[2019-09-20T22:02:00,506][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2019-09-20T22:02:00,521][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] [Consumer clientId=logstash-0, groupId=logstash] Successfully joined group with generation 13
[2019-09-20T22:02:00,522][INFO ][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] [Consumer clientId=logstash-0, groupId=logstash] Setting newly assigned partitions [cat.dev1.service.NJPW7DT004.SM1-0, cat.dev1.service.NJPW7DT004.SM1-1]

I expected to see the index 'test_boris' in the Kibana Management Is there something I am missing in the config?

Answer 1

The index won't be created until logstash processes events that end up in the index. So the root issue is likely that there's nothing coming in from kafka topic.