I can connect to elastic server using curl as shown below.
# curl --user "root:xxxxx" https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com/_aliases
I get the expected response and it means the credentials are correct.
{".kibana_3506402_root":{"aliases":{}},".opendistro_security":{"aliases":{}},".kibana_1":{"aliases":{".kibana":{}}}}
But the same credentials are not working in packetbeat
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:80"]
# Optional protocol and basic auth credentials.
protocol: "https"
username: "root"
password: "xxxxx"
As per logs, it is trying to connect on port 9200 even if port 80 is mentioned in the config file.
# tail /var/log/packetbeat/packetbeat
2020-03-01T13:56:03.234Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:9200)): Get https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:9200: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2020-03-01T13:56:03.234Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(https://search-testme-gvzxezayzzc4pcw2xcyvndb6jq.us-east-1.es.amazonaws.com:9200)) with 10 reconnect attempt(s)
How do I connect to AWS elastic using open distro through packetbeat? Similar configuration works without any problem if I use elastic cloud hosted by AWS.
There are 2 important points to remember:
1) You need "oss" (Open Source Version) of packetbeat for this to work.
2) Connect to 443 port
Detailed instructions can be found here...
http://oksoft.blogspot.com/2020/03/packetbeat-to-elastic-server-hosted-by.html