How to transfer data to a host with a username and password

Question

I have a problem. Current problem is that I want to transfer some data to a host with an username and password, but I keep getting the same error message. I would be very happy if you help.

My conf file:

input { 
   file { 
      path => "........../*.txt"
      start_position => "beginning"
      sincedb_path => "NUL" 
   }
}
filter {
............. 
}
output {
    elasticsearch { 
      hosts => "xx.xx.xxx.xxx:xxxx" 
      manage_template => false 
      index => "my_index_name"
      document_type => "my_index_name" 
      user => "my_user_name"
      password => "my_password" 
    }

Error message:

[WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elastic_user_name:[email protected]:xxxx/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '403' contacting Elasticsearch at URL 'http://xx.xx.xxx.xxx:xxxx/'"}

I also made changes to logstash.yml and elasticsearch.yml files as follows, but I got the same error.

elasticsearch.yml: xpack.management.elasticsearch.username: my_elastic_user_name xpack.management.elasticsearch.password: my_password

logstash.yml: xpack.monitoring.elasticsearch.username: my_elastic_user_name xpack.monitoring.elasticsearch.password: my_password

Answer 1

Receiving a HTTP 403 response code ("Forbidden") indicates that your user does not have permissions to index data to Elasticsearch (see this answer for the difference between Unauthorized (401) and Forbidden (403)).

Your user should have the following permissions

Setting up such a role is described here.

Please refer to the documentation about security privileges in order to adapt it to your use case.

For xpack monitoring you should create a user with the logstash_system built-in role.

I hope I could help you.