Python template safe substitution with the custom double-braces format

Question

I am trying to substitute variables in the format {{var}} with Python's Template.

from string import Template

class CustomTemplate(Template):
    delimiter = '{{'
    pattern = r'''
    \{\{(?:
    (?P<escaped>\{\{)|
    (?P<named>[_a-z][_a-z0-9]*)\}\}|
    (?P<braced>[_a-z][_a-z0-9]*)\}\}|
    (?P<invalid>)
    )
    '''

replacement_dict = {
    "test": "hello"
}

tpl = '''{
    "unaltered": "{{foo}}",
    "replaced": "{{test}}"
}'''

a = CustomTemplate(tpl)
b = a.safe_substitute(replacement_dict)

print(b)

The output:

{
    "unaltered": "{{foo",
    "replaced": "hello"
}

As you can see, the {{foo}} variable (which is not part of the replacement) has the closing brackets chopped off. I think it's the way the regex is written (the closing \}\})?

I want to solve this with Template, not with any other external libraries.

Answer 1

I'm not sure how you got this to work. On linux, in python 3.4.3 (I thought I got this working with some version of 2.7) I needed to make tpl a string

tpl = '''
    "unaltered": "{{foo}}",
    "replaced": "{{test}}"
'''

to avoid getting a TypeError

>>> tpl = '''
...     "unaltered": "{{foo}}",
...     "replaced": "{{test}}"
... '''
>>> a = CustomTemplate(tpl)
>>> a.template
'\n    "unaltered": "{{foo}}",\n    "replaced": "{{test}}"\n'
>>> b = a.safe_substitute(replacement_dict)
>>> b
'\n    "unaltered": "{{foo}}",\n    "replaced": "hello"\n'

When I do this, {{foo}} is unaltered.

I tried the above code, and it looks like the code does not actually work with python 2.7.6. I'll see if I can find a way to make it work with 2.7.6, since that seems to be a common version with recent linux distros.

update:

Looks like this was a known bug as of 2007. http://bugs.python.org/issue1686 Far as I can tell, it was applied to python 3.2 in 2010, and python 2.7 in 2014. As far as getting this to work, you can either apply the patch for issue 1686, or you can override safe_substitute() in your class with the actual source code from this patch https://hg.python.org/cpython/file/8a98ee6baa1e/Lib/string.py.

This code works in 2.7.6, and 3.4.3

from string import Template
class CustomTemplate(Template):
    delimiter = '{{'
    pattern = r'''
    \{\{(?:
    (?P<escaped>\{\{)|
    (?P<named>[_a-z][_a-z0-9]*)\}\}|
    (?P<braced>[_a-z][_a-z0-9]*)\}\}|
    (?P<invalid>)
    )
    '''

    def safe_substitute(self, *args, **kws):
        if len(args) > 1:
            raise TypeError('Too many positional arguments')
        if not args:
            mapping = kws
        elif kws:
            mapping = _multimap(kws, args[0])
        else:
            mapping = args[0]
        # Helper function for .sub()
        def convert(mo):
            named = mo.group('named') or mo.group('braced')
            if named is not None:
                try:
                    # We use this idiom instead of str() because the latter
                    # will fail if val is a Unicode containing non-ASCII
                    return '%s' % (mapping[named],)
                except KeyError:
                    return mo.group()
            if mo.group('escaped') is not None:
                return self.delimiter
            if mo.group('invalid') is not None:
                return mo.group()
            raise ValueError('Unrecognized named group in pattern',
                             self.pattern)
        return self.pattern.sub(convert, self.template)

replacement_dict = {
    "test": "hello"
}

tpl = '''{
    "escaped": "{{{{",
    "unaltered": "{{foo}}",
    "replaced": "{{test}}",
    "invalid": "{{az"
}'''

a = CustomTemplate(tpl)
b = a.safe_substitute(replacement_dict)

print (b)

results:

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import template
{
    "escaped": "{{",
    "unaltered": "{{foo}}",
    "replaced": "hello",
    "invalid": "{{az"
}
>>>