We have a scenario where we have potentially 100's of Linux systems in the field and we'd like to remotely manage the software deployed onto these systems. These systems are mobile, only connect to the network over WiFi and are in a fairly rough environment. They occasionally need to be replaced, requiring the same software and configuration to be deployed onto a new system in order to replace the existing one.
As the system moves around in and out of network coverage any updates to the system can be deployed to it remotely, however when the system is being installed/replaced it's not uncommon for it to be stationary in a location without any WiFi coverage. As a result, whatever we choose to do we'll need to have the option of deploying a new/replacement system using nothing more than a USB stick.
To make matters more complicated, each system consists of 2 or more computers on a local wired network. We'd like to be able to have 1 of those computers be a 'master', such that we only need to update that computer with the USB stick and any other computers on the local network (the 'slaves') would be automatically updated by the master. We'd also like to support the scenario where one of the slaves is replaced (already loaded with some minimally functional system) and when it connects to the 'master' it will be automatically updated.
I've been (briefly) looking at SaltStack and it looks like a good option when network connectivity is available (in this situation, the 'slaves' could just connect directly to the server and get their updates from there). However I'm not sure how well it will work for us in the USB stick deployment scenario. Ideally I'd like to be able go to the SaltStack server, enter the 'master' computer's minion ID somewhere and download a deployment image onto a USB stick that would not only allow us to configure the 'master' but also allow the 'master to automatically configure any connected 'slave' computers.
We're currently using an adhoc system that we've developed ourselves, but it's in desperate need of replacement. We're not adverse to doing some development ourselves on top of SaltStack to support this scenario, however I'd like to get a feeling for whether or not this is a sensible path to take or if we're better off staying with something custom.
Some other random bits of information that may be relevant: * The systems themselves involve several different CPU architectures: ARM, PowerPC and x86. * The systems aren't servers, they're field devices with somewhat limited resources. Ranging from a 200MHz PowerPC with 256MB of RAM up to an Intel Atom Bay Trail with 2GB of RAM. Storage ranges from 512MB up to several GB. * They are all running a Yocto-based Linux distribution.
There are four well acknowledged open source tools in the area of configuration management (so-called CAPS tools):
For your use-case I would draw your attention to Ansible, since it is primarily designed to operate in a agent-less fashion over SSH, storing its configuration in files normally mastered in an Git repository.
I'm pretty certain you could adapt any of these tools, but you might find it convenient to install ansible on a laptop and use this as your mobile workstation from which configuration runs are launched.