Python: dynamic json with string interpolation

Question

I created a class of functions that provision some cloud infrastructure.

response = self.ecs_client.register_task_definition(
containerDefinitions=[
                {
                    "name": "redis-283C462837EF23AA",
                    "image": "redis:3.2.7",
                    "cpu": 1,
                    "memory": 512,
                    "essential": True,
                },
...

This is a very long json, I show just the beginning.

Then I refactored the code to use a parameter instead of the hard coded hash, memory and cpu.

response = self.ecs_client.register_task_definition(
containerDefinitions=[
                {
                    "name": f"redis-{git_hash}",
                    "image": "redis:3.2.7",
                    "cpu": {num_cpu},
                    "memory": {memory_size},
                    "essential": True,
                },
...

I read the values of git_hash, num_cpu and memory_size from a config file prior to this code.

Now, I also want to read to entire json from a file.

The problem is that if I save {num_cpu} etc. in a file, the string interpolation won't work.

How can I extract the json from my logic and still use string interpolation or variables?

Answer 1

You can use Template from string.

{
    "name": "redis-${git_hash}",
    "image": "redis:3.2.7",
    "cpu": ${num_cpu},
    "memory": ${memory_size},
    "essential": true
}

from string import Template
import json

if __name__ == '__main__':
    data = dict(
        num_cpu = 1, 
        memory_size = 1,
        git_hash = 1
    )
    with open('test.json', 'r') as json_file:
        content = ''.join(json_file.readlines())
        template = Template(content)
        configuration = json.loads(template.substitute(data))
        print(configuration)

# {'name': 'redis-1', 'image': 'redis:3.2.7', 'cpu': 1, 'memory': 1, 'essential': True}

Opinion: I think the overall approach is wrong. There is a reason why this method is not as popular as others. You can separate your configuration into two files (1) a static list of options and (2) your compact changeable configuration, and compose them in your code.

EDIT: You can create an object which reads the configuration from a standard (static or changeable) JSON file FileConfig. And then compose them using another object, something line ComposedConfig.

This will allow you to extend the behaviour, and add, for example, a run-time configuration in the mix. This way the configuration from your JSON file no longer depends on the run-time params, and you can separate what is changeable from what is static in your system.

PS: The get method is just an example for explaining the composed behaviour; you can use other methods/designs.

import json
from abc import ABC, abstractmethod 


class Configuration(ABC):
    
    @abstractmethod
    def get(self, key: str, default: str) -> str:
        pass


class FileConfig(Configuration):

    def __init__(self, file_path):
        self.__content = {}
        with open(file_path, 'r') as json_file:
            self.__content = json.load(json_file)
            
    def get(self, key: str, default: str) -> str:
        return self.__content.get(key, default)


class RunTimeConfig(Configuration):
    def __init__(self, option: str):
        self.__content = {'option': option}
    
    def get(self, key: str, default: str) -> str:
        return self.__content.get(key, default)


class ComposedConfig:

    def __init__(self, first: Configuration, second: Configuration):
        self.__first = first
        self.__second = second

    def get(self, key: str, default: str) -> str:
        return self.__first.get(key, self.__second.get(key, default))


if __name__ == '__main__':
    static = FileConfig("static.json")
    changeable = FileConfig("changeable.json")
    runTime = RunTimeConfig(option="a")
    config = ComposedConfig(static, changeable)
    alternative = ComposedConfig(static, runTime)
    print(config.get("image", "test")) # redis:3.2.7
    print(alternative.get("option", "test")) # a