I am using CDK to deploy an EC2 instance that will run an application in a single node docker swarm cluster. This is not for critical production workloads, it is mostly for running side projects and experimenting. For the most part, it is working just fine and I'm able to access my app over the internet.
Here's my issue: when I redeploy the application it replaces the EC2 instance and all of the data is lost since it is using the instance's root volume to store data. I'm now trying to mount an EBS volume to the instance and mount docker volumes on that mounted EBS volume so that data is persisted between stack updates. Here is some information about application I am developing for more context:
The stack that I am deploying in docker swarm has the following services:
I would like to mount an EBS volume on traefik, postgres and redis, and probably just use the same volume for all three services to keep it simple.
I have been looking at the documentation for EBS here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html
I think that I need to do something like this in my UserData script:
# mount the EBS volume
sudo mkdir /data # make a directory on the EC2 machine
sudo mkfs -t xfs /dev/sda1 # create an xfs file system on /dev/sda1
sudo mount /dev/sda1 /data # mount the volume on the directory that was created
I think this is close to what I need, but it formats the volume's data each time I launch or replace the EC2 instance.
Should I be using fstab? I'm trying to add this to my UserData script:
sudo mkdir /data
echo "/dev/sda1 /data xfs defaults 0 0" >> /etc/fstab
This is still not persisting data. I'm testing this by adding a file to /data, redeploying and checking to see if the file exists once the EC2 instance has been replaced.
I am using CloudFormationInit scripts defined in my CDK stack to install docker, initialize a swarm cluster, download a stack.yml file and deploy it to the swarm cluster. I then create a Route 53 record that points to the public IP of the EC2 instance.
Here is a link to the CDK construct that I am working on for running my Django application in docker swarm on EC2: https://github.com/briancaffey/django-cdk/blob/main/src/docker-ec2.ts
You almost got it - you just mount it and that's it. Formatting it does indeed wipe the data, the solution is simply to skip that step.
The docs you link to address this:
(Conditional) If you discovered that there is a file system on the device in the previous step, skip this step. If you have an empty volume, use the mkfs -t command to create a file system on the volume.
Warning. Do not use this command if you're mounting a volume that already has data on it (for example, a volume that was created from a snapshot). Otherwise, you'll format the volume and delete the existing data.
So the following should work:
# mount the EBS volume
sudo mkdir /data # make a directory on the EC2 machine
sudo mount /dev/sda1 /data # mount the volume on the directory that was created