I'm currently building a development environment based on VS Code DevContainers. To build all configurations of the software I would need roughly 50GB of tooling dependencies. This in and of itself is not an issue, as starting a large image with Docker has very little overhead. However, different versions of our software require different dependencies, leading to many large images quickly taking up all disk space.
How could I go about effectively distributing the dependencies to all developers? In my case, most developers have little experience or interest in managing the dependencies. Ideally, dependencies are automatically managed by some tool.
The operating system is Linux (WSL2). Our build system is currently just one large Python project (100k+ LOC) which depending on the parameters builds everything. The compilers and other tools are all defined via environment variables.
My initial thought was to create a small DevContainer that can run the Python build environment. Then I've prepended all system calls to the tools with docker exec ... container sh -c '<original call>'. Before running any command I pull the image and run the container with the corresponding tools. This works surprisingly well, even in our CI/CD environment, with very few modifications to our large code base. However, it is slow. The build time doubled from 4 to 8 minutes.
VScode DevContainers can have mounts so can link local folders from your host to your DevContainer. So you have the 50GB dependencies just on your host system.
devcontainer.json example
...
"mounts": [
"source=MyDependencies,target=/{{My folder for the dependencies inside the devContainer}},type=volume"
]
...
Create .devcontainer/devcontainer.json and .devcontainer/Dockerfile
in the devcontaoner.json add
...
{
"build": {
"dockerfile": "Dockerfile",
"args": { }
}
}
...
In the Dockerfile define your own base-image FROM you can use on DevContainer base-image https://containers.dev/implementors/reference/#labels or create a custom Docker-Image.