Use the last rather than the first stored value when using multiple sources with boost::program_options

Question

My goal is to keep config values which don't change most of the time in one or more INI-files and read the remaining parameters from the command line. I am using boost::program_options to achieve this.

Basically,

options_description options;
variables_map vm;
ifstream ifs("config.ini");
store(parse_config_file(ifs, options), vm);
// !!! Does not overwrite values from config.ini
store(command_line_parser(argc, argv).options(options).run(), vm);

This works nicely so far as long as the parameters provided in the INI and the command line don't overlap. In case they overlap, the desired and logical behavior would be for the parameters from the command line to overwrite anything previously read from the INI files. That does not work, though, due to Boost's default behavior of using the first encountered value and ignoring any others after that.

Do you have any ideas how to bypass that limitation?

A complete working example of what I am doing:

#include <iostream>
#include <fstream>
#include <vector>
#include <boost/program_options.hpp>

using namespace std;
using namespace boost::program_options;

// Parsed values will be stored in these
std::vector<std::string> inputFiles;
int numThreads;
bool enableX;

void parseOptions(const int argc, const char* argv[])
{
    //
    // Define our config parameters
    //
    options_description basicOptions("Main options");
    basicOptions.add_options()
        ("help,h", "Show full help.")
        ("version,V", "Show version.")
        ("config,c",
         value<vector<string>>()->multitoken()->composing(),
         "Path of a configuration INI file. Can be used repeatedly.\n"
         "The available config parameters are listed in the full help info."
        )
        ("inputFile,i",
         value<vector<string>>()->multitoken()->composing()->required(),
         "Input file(s). Can be used repeatedly."
        );
    options_description configOptions("Configuration");
    configOptions.add(basicOptions);
    configOptions.add_options()
        ("numThreads",
         value<>(&numThreads)->default_value(1),
         "Number of processing threads."
        )
        // ... snip ...
        ("enableX",
         value<>(&enableX)->default_value(true),
         "Whether to enable X."
        );

    //
    // Parse
    //
    variables_map vm;

    // Parse the minimal set of command line arguments first
    parsed_options parsedCL = command_line_parser(argc, argv).
        options(basicOptions).allow_unregistered().run();
    store(parsedCL, vm);

    // Load configuration from INI files
    if (vm.count("config"))
    {
        for (int i = 0; i < vm["config"].as<vector<string>>().size(); i++)
        {
            ifstream ifs(vm["config"].as<vector<string>>()[i].c_str());
            store(parse_config_file(ifs, configOptions), vm);
        }
    }

    // Parse and store the remaining command line arguments
    // !!! store() does not overwrite existing values !!!
    store(command_line_parser(
              collect_unrecognized(parsedCL.options, include_positional)
          ).options(configOptions).run(), vm);

    // Finally, check that that the provided input is valid
    notify(vm);
}

int main(const int argc, const char* argv[])
{
    try
    {
        parseOptions(argc, argv);
        cout << "enableX: " << enableX << endl;
    }
    catch (exception e)
    {
        cerr << e.what() << endl;
    }
    return 0;
}

If you run the above program with --config config.ini --enableX false, with the content of config.ini being

numThreads = 4
enableX = true

it outputs "enableX: 1", instead of "enableX: 0" – the desired value from the command line.

In a hackish attempt at a fix, I have tried simply removing the conflicting parameters from the variable_map before storing any new ones:

void store_with_overwrite(const parsed_options& parsed, variables_map& vm)
{
    for (const option& option : parsed.options)
    {
        auto it = vm.find(option.string_key);
        if (it != vm.end())
        {
            vm.erase(it);
        }
    }
    store(parsed, vm);
}

This does not work since Boost stores the first encountered value in a private m_final member variable as well, which gets used even if the parameter has been removed from the map.

The only solution I can think of now is to parse the command line arguments first and then remove any conflicting values in parsed_options from INI files before store()-ing them. The downside is that I would have to manually keep track of options that use multitoken() and composing(), which must not be removed.

There has to be a better way. Do you have any suggestions?

Answer 1

Since the first stored values are the final ones that stick, which is also what I want the command line ones to be, the solution is actually simple: parse and store everything from the command line first.

In general, store values from different sources in a reverse order compared to the order you want them to "overwrite" each other.

variables_map vm;

// Parse the command line arguments first
parsed_options parsedCL = command_line_parser(argc, argv).
    options(configOptions).run();
store(parsedCL, vm);

// Load configuration from INI files
if (vm.count("config"))
{
    for (int i = 0; i < vm["config"].as<vector<string>>().size(); i++)
    {
        ifstream ifs(vm["config"].as<vector<string>>()[i].c_str());
        store(parse_config_file(ifs, configOptions), vm);
    }
}

// Finally, check that that the provided input is valid
notify(vm);