How to set logstash configuration output to use both HTTP & other HTTPS endpoints?

Question

I want to send data to 2 endpoint from logstash, one of which is HTTP endpoint & other is HTTPS.

I tried putting username & password for HTTPS endpoint in url itself but logstash is taking those fields [username & password] for the other endpoint also.

my current output field if like:

output {
    elasticsearch{
        index => "index_name"
        document_id => "%{index_id}"
        hosts => ["https://elastic:pass@clusterid.asia-northeast1.gcp.cloud.es.io:9243",
                  "http://127.0.0.1:9200"]
                }
      }

Getting this message in logs:

Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@clusterid.asia-northeast1.gcp.cloud.es.io:9243/, https://elastic:xxxxxx@127.0.0.1:9200/]}}

And this:

[logstash.agent] Failed to execute action {:id=>:"cloud-elastic", :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<cloud-elastic>, action_result: false", :backtrace=>nil}

Answer 1

please try using different elasticsearch output for https and http,below settings

if "Https" in [tag]{
    elasticsearch {
                hosts => [ "https://elastic:pass@clusterid.asia-northeast1.gcp.cloud.es.io:9243" ]
                user => "${ES_USER:admin}"
                password => "${ES_PWD:admin}"
                ssl => true
                ssl_certificate_verification => false
                cacert => "${CERTS_DIR}/certs/ca.crt"
                index => "%{[docInfo][index]}"
                action => "index"
              }
            } else {
               elasticsearch {
                hosts => [ "http://127.0.0.1:9200" ]
                index => "%{[docInfo][index]}"
                action => "index"
              }
}

In .bashrc file

set the below environment variables

export ES_USER=elastic
export ES_PWD=pass
export CERTS_DIR=/home/abc