Search code examples
dockeramazon-elastic-beanstalkyoutrackamazon-efs

Can a single-instance Elastic Beanstalk deploy be configured to stop the current container before starting the new one?


I'm working on upgrading an old version of YouTrack to the latest Docker image. The image is stateful, in that it stores its configuration and data on an EFS volume mounted to the container.

I'm running into a problem while trying to deploy the version upgrade, which manifests as:

Command failed on instance. Return code: 1 Output: 
2500dea09a203c18d36aa997159750e8f1b37a7ad0437bc109a5073f38ce24be Docker container quit unexpectedly 
after launch: Docker container quit unexpectedly on Mon Jul 6 23:43:26 UTC 2020: YouTrack is already 
running. Check snapshot logs for details. Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh 
failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.

(The referenced log shows only the same error followed by some 'Activity Failed' entries.)

It seems as though:

  • EB loads the new image to the instance.
  • It leaves the old image running while it fires up the new one.
  • I'm guessing that the new image detects that another version of YouTrack has locked the database via the mounted filesystem, and quits.

I've read in this answer that rolling updates can be configured as immutable for multi-instance environments, which forces a new instance to be created on each deploy, which sounds as though it would solve the problem, but I don't have that option for my single-container single-instance environment (Rolling Update Type can only be 'Disabled').

I've tried eb ssh into the instance to stop the old container, but something is instantly starting it again - have tried both docker update --restart=no <container> per this answer, and the recommended stop command.

Is there some way to configure EB to stop the old container before starting the new one (or to use a new instance, which is less preferable but would get the job done)?


Solution

  • Success! Not a permanent configuration but a good enough workaround - this article gave the answer on how to stop the container manually, then it was just a case of running a deploy.

    Steps were:

    1. SSH to the ElasticBeanstalk instance with eb ssh -i <instance id>.
    2. sudo stop eb-docker to tell Upstart to stop the container.
    3. docker ps to verify that the container's no longer running.
    4. Carry out the EB deploy as normal.