Here is what I do:
I choose Launch Instance
from my ec2 dashboard
I select Ubuntu Server 12.04.3 LTS - 64-bit
from the AMI list
I choose t1.micro
as my instance type
I don't change anything on step2 (Configure Instance Details)
On step 4, I increase the size of the volume from 8
to 200
I click on Review and Launch
and run the instance.
Now I can not ssh to the server although its state is running
and Status Checks is 2/2 checks passed
. I don't have any problem with doing the same steps except for increasing the volume size part. Any idea why this happens?
New EC2 instances with an EBS root volume are started from bootable hard drive snapshots with some metadata, collectively known as an Amazon Machine Image (AMI).
The question prompted me to realize something: given the fact that we're starting off with an existing hard drive image, with an established filesystem on it, it logically follows that no matter how large of a disk that filesystem is copied onto before being bound to our new machine, the filesystem had already been created, and it wouldn't normally be aware of the extra amount of space that happened to be available on the disk, above and beyond its original size.
And yet, I've selected larger-than-default size values for the root disk with the Ubuntu 12.04 LTS AMI and never given a moment's though to the fact that the amount of space I provisioned is "magically" available, when, logically, it shouldn't be. The filesystem should still be 8 GB after bootup, because it's a copy of a filesystem that was originally 8 GB in size, and all of its internal structures should still indicate this.
The only possible conclusion is that the snapshot we're initially booting must actually contain code to automatically grow its own filesystem to fill the disk that it wakes up to find itself running on.
This... turns out to be true. From an early write-up about EC2/EBS by Eric Hammond, describing how to get a larger root volume:
There’s one step left. We need to resize the file system so that it fills up the entire 100 GB EBS volume. Here’s the magic command for ext3. In my early tests it took 2-3 minutes to run. [Update: For Ubuntu 11.04 and later, this step is performed automatically when the AMI is booted and you don’t need to run it manually.] [emphasis added]
But, you apparently do need to wait for it, and the bigger the disk, the longer the wait. The instance reachability check sounds like a ping test. If so, it's conceivable that the network stack could be up and responsive but sshd might not yet available to accept connections during the resize operation, which would cause the "connection refused" response -- which is an active refusal by the IP stack because the destination socket isn't listening.
The instance reachability check doesn't so much mean the instance is "ready" as much as it means it is "on its way up, or up, and not on its way down."