For multiple projects using Docker, use Multiple VMs or Single Host with multiple containters

Question

Suppose I had three apps that are currently hosted at Digital Ocean or AWS. Each of them use at least one VM for the database and one or more VMs for the web app.

Now let's say that I wanted to get one dedicated server at OVH with 64GB of RAM and use docker to deploy these apps. Each project would have its own docker-compose file. I'm thinking of two ways of doing this:

1. Install VMWare Esxi on the server, create one VM for each project and deploy docker containers for the web and database.
2. Just install Ubuntu as the host OS and manage containers for all apps using separate network entry points(IPs) for each project.

Would I be wasting too much of the server resources going for the first choice? Would I be overcomplicating my infrastructure by going for the second?

I understand both are valid choices, but what would be the better/suggested way?

Thanks for the help!

Answer 1

1. will need a full VM for each app with all the memorysharing and I/O stuff. You may use memory ballooning with virtual box (ESXi should have such a feature too, maybe named different). And within every VM you'll have have the docker stack included.
2. If you use a native OS you'll need the docker stack only once.

What OS do you use within your docker images? An 200MB Ubuntu? Or a 5MB Alpine? If you choose Alpine as your host OS and/or your image OSes you'll be able to keep your "container overhead" much smaller

It depends what system services your apps need, like cron, upstart, ... how much resources does the app need? Is it JVM based app that needs an own JVM in every container? Etc...

At first hand I would plan an Alpine Host with Alpine docker images. If there are Apps that realy need Ubuntu images you can just use it in the image for that specific app.

Have also a look at Docker vs. VM