How to set up autosearch nodes in Elasticsearch 6.1

I have created cluster of 5 nodes in ES 6.1. I am able to create cluster when I added line with all ip addresses of other nodes into configuration file elasticsearch.yaml as discovery.zen.ping.unicast.hosts. It looks like this:

discovery.zen.ping.unicast.hosts: ["10.206.81.241","10.206.81.238","10.206.81.237","10.206.81.239"]

When I have this line in my config file, everything works well.

ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.206.81.241            9          54   0    0.03    0.05     0.05 mi        *      master4
10.206.81.239           10          54   0    0.00    0.01     0.05 mi        -      master1
10.206.81.238           14          54   0    0.00    0.01     0.05 mi        -      master3
10.206.81.240           15          54   0    0.00    0.01     0.05 mi        -      master5
10.206.81.237           10          54   0    0.00    0.01     0.05 mi        -      master2

When I added discovery.zen.ping.multicast.enabled: true elasticsearch will not start.

I would like to have more nodes and if I will have to configure each file separately and add new address to each configuration every time, it is not proper way. So is there any way how to set up ES6 to find new nodes automatically?

EDIT:

journalctl -f output:

led 08 10:43:04 elk-prod3.user.dc.company.local polkitd[548]: Registered Authentication Agent for unix-process:23395:23676999 (system bus name :1.162 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
led 08 10:43:04 elk-prod3.user.dc.company.local systemd[1]: Stopping Elasticsearch...
led 08 10:43:04 elk-prod3.user.dc.company.local systemd[1]: Started Elasticsearch.
led 08 10:43:04 elk-prod3.user.dc.company.local systemd[1]: Starting Elasticsearch...
led 08 10:43:04 elk-prod3.user.dc.company.local polkitd[548]: Unregistered Authentication Agent for unix-process:23395:23676999 (system bus name :1.162, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
led 08 10:43:07 elk-prod3.user.dc.company.local systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
led 08 10:43:07 elk-prod3.user.dc.company.local systemd[1]: Unit elasticsearch.service entered failed state.
led 08 10:43:07 elk-prod3.user.dc.company.local systemd[1]: elasticsearch.service failed.

Solution

Basically you should have "stable" nodes. What i mean is that you should have IPs which are always part of cluster

discovery.zen.ping.unicast.hosts: [MASTER_NODE_IP_OR_DNS, MASTER2_NODE_IP_OR_DNS, MASTER3_NODE_IP_OR_DNS]

Then if you use autoscaling or add nodes they must "talk" to that ips to let them know that they are joining the cluster.

You haven't mentioned anything about your network setup so i can say you for sure what is wrong. But as I recall unicast hosts is recommended approach

PS. If you are using azure, there is feature called VM scaleset I modified template to my needs. Idea is that by default I am always using 3 nodes, and if my cluster is loaded scale set will add dynamically more nodes.