X-Pack & Logstash - No output to elasticsearch

I have an installation of X-Pack, Elasticsearch & Logstash. Logstash is alreay reporting to elasticsearch for the monitoring part of X-Pack.

Now I want logstash to pass through any beat that comes to the pipe to elasticsearch.

I followed this guide here and created the mentioned user & role. I additionally created a role for each beat type that has the mentioned privileges and assigned the roles to the same user, (e.g. one for filebeat-*), using the information from here.

My logstash.conflooks the following:

input {
  beats {
   port => 5044
  }
}
output {
  elasticsearch {
   hosts => ["http://localhost:9200"]
   index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
   document_type => "%{[@metadata][type]}"
   user => "user"
   password => "pass-word"
 }
  file {
   path => "C:\temp\%{[@metadata][beat]}-%{+YYYY.MM.dd}.log"
 }
}

I am getting files created (e.g. for filebeat) but the indices in elasticsearch are not created and no data appears in elasticsearch.

Does anyone see if there is something wrong in the configuration? I am struggling since hours to find an error/solution.

Publishing beats directly to elasticsearch (without Logstash) using the mentioned user is working (e.g. using metricbeat).

Also I do not see something strange in the log, simply the output to elastic is missing:

07:52:53.807 [[main]>worker0] DEBUG logstash.pipeline - filter received {"event"=>{...}}
07:52:53.807 [[main]>worker0] DEBUG logstash.pipeline - filter received {"event"=>{...}}
07:52:53.807 [[main]>worker0] DEBUG logstash.pipeline - output received {"event"=>{...}}
07:52:53.807 [[main]>worker0] DEBUG logstash.pipeline - output received {"event"=>{...}}
07:52:53.870 [[main]>worker0] DEBUG logstash.outputs.file - File, writing event to file. {:filename=>"C:/temp/filebeat-2017.04.11.log"}
07:52:53.870 [[main]>worker0] DEBUG logstash.outputs.file - File, writing event to file. {:filename=>"C:/temp/filebeat-2017.04.11.log"}
07:52:53.870 [[main]>worker0] INFO logstash.outputs.file - Opening file {:path=>"C:/temp/filebeat-2017.04.11.log"}
07:52:53.870 [[main]>worker0] DEBUG logstash.outputs.file - Starting stale files cleanup cycle {:files=>{"C:/temp/filebeat-2017.04.11.log"=>#>}}
07:52:53.870 [[main]>worker0] DEBUG logstash.outputs.file - 0 stale files found {:inactive_files=>{}}

Thank you already!

UPDATE:

If I change the password, I get the following error in the logstash log:

08:22:33.877 [[main]>worker1] ERROR logstash.outputs.elasticsearch - Got a bad response code from server, but this code is not considered retryable. Request will be dropped {:code=>401, :response_body=>"{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"unable to authenticate user [beat] for REST request [/_bulk]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\"security\\" charset=\\"UTF-8\\"\"}}],\"type\":\"security_exception\",\"reason\":\"unable to authenticate user [beat] for REST request [/_bulk]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\"security\\" charset=\\"UTF-8\\"\"}},\"status\":401}", :request_body=>"{\"index\":{\"_id\":null,\"_index\"

So it seems to be actually communicating with elasticsearch, but the index does not get created. Unfortunately I have no statement in the elasticsearch log.

Solution

Found the answer on my own, for those struggling with a similar problem:

I had some weird settings in my elasticsearch.yml regarding action.auto_create_index that prevented the index from being created.

This became visible after changing the log level of elasticsearch to debug the way it was mentioned here.

As those were not needed I removed them and now everything works as expected.