How to get rid of quotes in strings when using yaml.dump

Question

I am trying to use ruamel.yaml to generate cloudformation YAML templates from ordered dictionary. I need to prefix some of the strings with "!Sub" which is a cloudtemplate function reference in a way that it is not enclosed in quotes with the rest of the strings, e.g.:

Type: AWS::Glue::Job
Properties:
  Role2: !Sub 'arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}'

Test code I started with looks like this:

from ruamel import yaml
from collections import OrderedDict

def ordered_dict_presenter(dumper, data):
    return dumper.represent_dict(data.items())

test = OrderedDict(
    {
        "Type": "AWS::Glue::Job",
        "Properties": {
            "Name": "test_job",
            "Role": "!Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}",
            "Role2": "Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}",
        },
    }
)
yaml.add_representer(OrderedDict, ordered_dict_presenter)

print(yaml.dump(test, default_style=None, default_flow_style=False))

It outputs this yaml:

Type: AWS::Glue::Job
Properties:
  Name: test_job
  Role: '!Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}'
  Role2: Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}

It seems like when a string starts with non-alphabetic character it is automatically quoted. I tried custom representers to get rid of quotes but without any luck so far. How can I output this?

Answer 1

First of all you really shouldn't be using the old API anymore, there has been a new one, which allows much more control over what you are doing for several years now.

Second, you cannot put something that looks like a tag inside a string and not get quotes, during loading that would load like a tag and you might want that, but that would make an infinite number of strings (all those starting with !) non-representable in YAML.

If you don't know where to start, the thing to do is try and round-trip (load then dump) your required result:

import sys
import ruamel.yaml

yaml_str = """\
Type: AWS::Glue::Job
Properties:
  Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}
"""

yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)

this gives:

Type: AWS::Glue::Job
Properties:
  Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}

This confirms that ruamel.yaml can create the output. Now you only need to do that from scratch. For which you can inspect data and especially data['Properties']['Role2'] (if things don't round-trip that often but not necessary, means you cannot generate what you want, but find out how to do it in that case can be harder).

print(type(data['Properties']['Role2']))

This prints:

Type: AWS::Glue::Job
Properties:
  Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}
<class 'ruamel.yaml.comments.TaggedScalar'>

Now you only have to find out how TaggedScalar works ( source is in comments.py):

import sys
import ruamel.yaml

data = {}
data['Properties'] = props = {}
props['Role2'] = ruamel.yaml.comments.TaggedScalar('arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}', style=None, tag='!Sub')

yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
yaml.dump(data, sys.stdout)

giving:

Properties:
  Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}

Alternatively you can specify style="'" or style='"' if you do want to have quotes.