Can I use multiple elastic search hosts in Spring Bean configuration for multiple elastic search indexes

Question

I have a Java API to insert documents to three elastic indexes and it is working. I want to change one index's host in API. Normally, EsConfig file and ElasticSearchTemplate code is ;

public class EsConfig {
    @Value("${elasticsearch.host}")
    private String EsHost;
   @Value("${elasticsearch.port}")
   private int EsPort;
   @Value("${elasticsearch.clustername}")
   private String EsClusterName;

   @Bean
   public Client client() throws Exception {
      Settings settings = Settings.builder()
            .put("cluster.name", EsClusterName)
            //.put("index.max_result_window", 4000000)
            .build();

      TransportClient client = new PreBuiltTransportClient(settings)
            .addTransportAddress(new 
      TransportAddress(InetAddress.getByName(EsHost), EsPort));
      return client;
     }

     @Bean
     public ElasticsearchTemplate elasticsearchTemplate() throws Exception {
          ElasticsearchTemplate elasticsearchTemplate = new ElasticsearchTemplate(client());
          return elasticsearchTemplate;
      }

}

I want to configure this structure to use two hosts together. Is it possible in this structure or should I completely change and remove singleton bean structure?

Answer 1

Elasticsearch client api allows you to configure multiple host names in the below fashion but unfortunately, they do not work as expected.

According to this LINK,

The TransportClient connects remotely to an Elasticsearch cluster using the transport module. It does not join the cluster, but simply gets one or more initial transport addresses and communicates with them in round robin fashion on each action (though most actions will probably be "two hop" operations).

TransportClient client = new PreBuiltTransportClient(Settings.EMPTY)
        .addTransportAddress(new TransportAddress(InetAddress.getByName("host1"), 9300))
        .addTransportAddress(new TransportAddress(InetAddress.getByName("host2"), 9300));

What you can do perhaps is go ahead and implement a Spring Boot concept of Profiles where you can create multiple application.properties. Below is one of the ways to achieve this.

If I have two different hosts/environments for e.g. dev and prod, I would end up creating three application properties file (Two for the environments, one properties which would mention which environment you'd want to select).

application-dev.properties

elasticsearch.clustername = mycluster_name
elasticsearch.host = mydev.abc.com          <-- Configure the name of host 1
elasticsearch.port = 9300

application-prod.properties

elasticsearch.clustername = mycluster_name_2
elasticsearch.host = myprod.abc.com         <-- Configure the name of host 2
elasticsearch.port = 9300

application.properties

spring.profiles.active=dev                 <-- Configure which of the above properties do you want to start application with

Now when you run the application as spring boot, it would end up starting the dev environment.

Note that I'm assuming that both these nodes are in different clusters and not part of the same cluster. The reason I specify this is because, elasticsearch internally would go ahead and update the replica shards of other nodes if it receives a new/updated document. Being in same cluster, it wouldn't matter which host in a cluster you are pointing to.

Let me know if this is what you are looking for.