We are using Multiple Docker environment on Elastic BeanStalk with 2 instances running and we think there are some problem with one of the instances. What is the proper way to replace that particular instance?
I am thinking of one of the following:
The goal is to minimize down time.
If your goal is to minimize downtime then using the second option is recommended - i.e. increase the minimum number of instances to 3, wait for it to be healthy and then terminate the problematic instance. Before decreasing it back to 2, terminate the problematic instance. This is because automatic scaling will depend on EC2 health by default and even though your application will be unhealthy, your EC2 instance may be healthy. You might end up losing a healthy instance. So increase min size to 3, terminate the problematic instance and then scale back down to 2.
Also if you are not already using it, I would recommend you to use the enhanced health reporting feature which was launched by Elastic Beanstalk in August 2015. Details on enhanced health reporting can be found here. With enhanced health reporting you can get detailed health of environment and each instance. You can get a health status, color, system metrics (CPU, memory), application latency, error rate, failure rate and a list of causes giving you detailed health descriptions of your environment and instance health. This can be additional information for you to determine if your newly launched instance is healthy or not. Read a quick intro to this feature here.
If you plan to update your environment to enhanced health reporting keep in mind the instances in your environment will be replaced and hence if you are sensitive to downtime then it is better to scale up by 1 instance and then turn on rolling updates before switching your health system type to "enhanced". Read more about rolling updates here.