YAML-serialized data for multiple config files with Jinja2

Question

I've already been able to load data from a YAML document into a configuration file.

YAML:

id: 1
x: key1
y: key2

Python script:

#Import necessary functions from Jinja2 module
from jinja2 import Environment, FileSystemLoader

#Import YAML module
import yaml

import sys

ALLOWED_ARGS = set(['template', 'data'])

def print_usage():
    print('Wrong arguments: Usage: configplate.py --template=<template.jinja> --data=<data.yaml>')

def load_args(args):
    out = {}
    for arg in args:
        if arg == __file__: #ignore the filename argument
            continue
        args_splited = arg.split('=')
        out[args_splited[0].lstrip('-')] = args_splited[1]
    if set(out.keys()) != set(ALLOWED_ARGS):
        print_usage()
        raise ValueError('Required argument not present.')
    return out

def make(template, data):
    #Load data from YAML into Python dictionary
    config_data = yaml.load(open(data))

    #Load Jinja2 templates
    env = Environment(loader = FileSystemLoader('.'), trim_blocks=True, lstrip_blocks=True)
    template = env.get_template(template)

    #Render the template with data and print the output
    return template.render(config_data)

if __name__ == '__main__':
    if len(sys.argv) != 3:
        print_usage()
    else:
        args = load_args(sys.argv)
        res = make(args['template'], args['data'])
        print(res)

Config file:

<add name="Cloud" connectionString="{{ x }}" />

The issue I'm facing now is that I can't load the same YAML to multiple config files. I have a second config file that I would like to fill at the same time.

Config file 2:

<add name="Cloud" connectionString="{{ y }}" />

How can I add this in the same python script? I also do not need to set the config files on the console, I can simply put their path into the script.

Answer 1

Since you want to process multiple templates, you need to make your load_args capable of handling multiple --template= arguments/options. I adapted your load_args to return a dict with values that are lists, but the check on number of arguments allowed (multiple templates, single YAML file) is crude.

For processing your commandline you should look at the standard arparse module, where you can indicate per argument/option that it can occur multiple times.

Since you are going to reuse the data from YAML, load it only once, outside of the call make(). There is no excuse for not using safe_load() instead of the documented unsafe load().

With:

#Import necessary functions from Jinja2 module
from jinja2 import Environment, FileSystemLoader

#Import YAML module
import yaml

import sys

ALLOWED_ARGS = set(['template', 'data'])

def print_usage():
    print('Wrong arguments: Usage: configplate.py --template=<template.jinja> --data=<data.yaml>')

def load_args(args):
    out = {}
    for arg in args:
        if arg == __file__: #ignore the filename argument
            continue
        args_split = arg.split('=', 1)
        out.setdefault(args_split[0].lstrip('-'), []).append(args_split[1])
    if (set(out.keys()) != set(ALLOWED_ARGS)) or len(out['data']) != 1:
        print_usage()
        raise ValueError('Required argument not present.')
    return out

def make(template, data):
    #Load Jinja2 templates
    env = Environment(loader = FileSystemLoader('.'), trim_blocks=True, lstrip_blocks=True)
    template = env.get_template(template)

    #Render the template with data and print the output
    return template.render(data)

sys.argv=[
    __file__,
    '--template=config1.jinja',
    '--template=config2.jinja',
    '--data=data.yaml'
]

if __name__ == '__main__':
    if len(sys.argv) < 4:
        print_usage()
    else:
        args = load_args(sys.argv)
        # Load data from YAML into Python dictionary **once** and **safely**
        config_data = yaml.safe_load(open(args['data'][0]))
        for template_file in args['template']:
            res = make(template_file, config_data)
            print(res)

And the appropriate files you get:

<add name="Cloud" connectionString="key1" />
<add name="Cloud" connectionString="key2" />

---

When you import argparse you can drop AllOWED_ARGS, print_usage and load_args, and do then do the following:

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--template', action='append', nargs='+', required=True)
    parser.add_argument('--data', required=True)
    args = parser.parse_args()

    # Load data from YAML into Python dictionary **once** and **safely**
    config_data = yaml.safe_load(open(args.data))
    for template_file in args.template:
        res = make(template_file, config_data)
        print(res)

---

(BTW look up the conjugation of to split, there is no such thing as "splited" (or "splitted"), i.e. something "has been split", )