Search code examples
deploymentbuildinstallationenvironmentbuild-system

A meta-build solution managing multiple projects


I'm looking for an advice about the software that did not catch my eye during a few days of searches.

Is there any software solution for deployment of multiple projects at once that operates at the level of source-building package manager (like ports, portages, or nix), but could be resided locally?

As for details, we have few loosely connected software projects with the following traits:

  • Projects are written mostly on C/C ++ and Python, but more languages are represented as a minor mixture (Haskell, Rust, Perl)
  • Projects are grouped together within git superprojects with certain build/deployment presets carefully tuned for particular environments (and adapted for particular purposes by subset of boolean-like options)
  • We have already done with well-elaborated CMake build scripts for C++ projects that do support options, build configurations, exporting targets, and so on. It would be expensive to switch from it now.
  • We are forced to deal with various Linux distributions (from Gentoo and Ubuntu to Debian and CentOS).

We need an unifying build&deployment tool for various environments. The CMake does not integrate well with non-compiling languages (e.g. does not natively support local Python installations via virtualenv).

Instead of changing the things we already developed, I would like to use them in the manner the OS package manager does. For my vision, it should be something pretty similar to the so-called meta-build tool. In fact, the Gentoo Portages are pretty close:

  • Easy customization with simple boolean options (useflags, profiles)
  • Delegation of the building procedures to the reliable and customizable tools designed and well-tested especially for this purpose (CMake, autotools, Bazel, etc.)
  • Offers an ability to change the target compiler installation and specify the building process in a clear declarative way with a standardized instructions set. Portages can not be ran locally, though (and have other unrelated flaws).

I have to become very confident before switching the whole build system to something like Meson or Bazel or whatever else I could find up to the moment.

Update

To be more specific, I could refer to what we have done up to the moment. The one of the superprojects we're maintaining deals with particular scientific experiment:

  • All the sub-projects are listed as git submodules
  • A simple BASH-script maintaining the entire build-install-deploy lifecycle
  • The presets referring particular settings for a concrete environment are represented as BASH sources and are included by this maintaining shell script.

As few days more had come with no result, I coming up with intention to write my own solution based on experience we already gained due to these shell scripting activity.


Solution

  • For a similar task Bob was developed some time ago. If you still need a build system it might be worth a look. There are some small tutorials available as well as a complex basement example build a linux from scratch.

    Basically it is just a environment for a controlled execution of bash scripts with dependency and variant handling, CI integration, IDE support,..