How to preserve the order of Key Value in YAML file using python (not alphabetical order)

Question

I have the python code below for generating (dumping) a YAML document to a file.

import yaml
import os
device_name = "NN41_R11"
ip = "10.110.11.11"
port = "2022"
def genyam():
    data  = {
         "testbed" : {
            "name"  : "boot_ios"},

        "devices"  :  {
            device_name  :  {
                "type"  : "IOS",
                "connections"  : {
                    "defaults" : {
                        "class"  :  "con.con",
                    "a"  :  {
                        "protocol" : "telnet",
                        "ip" : ip,
                        "port" : port,
                    }
                    }
                }
            }
        }
        }

    with open('/tmp/testbed.yaml', 'w') as outfile:
        yaml.dump(data, outfile, default_flow_style=False)`

which generates the following YAML file

devices:
  NN41_R11:
    connections:
      defaults:
        a:
          ip: 10.110.11.11
          port: '2022'
          protocol: telnet
        class: con.con
    type: IOS
testbed:
  name: boot_ios

Though the key value indentation is correct it's not generating in right order. I would like to have testbed first & then devices however it's opposite now. I am suspecting it's dumping in alphabetical order. NN41_R11 is again a dictionary which contains type & connections (type & connections are generated at same level but need first type:IOS and under that connections). Looking for ordered dump basically

The generated YAML document should be like the following:

testbed:
    name: "boot-ios"
devices:
    NN41_R11:
        type: IOS
        connections:
            defaults:
                 class: 'con.con'
            a:
              protocol: telnet
              ip: 10.110.11.11
              port: 2022

Answer 1

I recommend you look at ruamel.yaml (disclaimer: I am the author of that package), it is specifically designed to preserve order of keys when loading and dumping (i.e. round-tripping) YAML documents and can also easily be used to generate YAML documents with your specifics on the fly.

You'll have to somehow order your key-value pairs in your source, as although there is order in the Python source this is not preserved in the dict with name data. The omap type (i.e. ruamel.yaml.comments.CommentedMap) can be initialised with list of tuples, but I often find it easier to use step-by-step assignment.

To get double and single quotes around those strings that don't need it use the dq (i.e. ruamel.yaml.scalarstring.DoubleQuotedScalarString) resp. sq (i.e. ruamel.yaml.scalarstring.SingleQuotedScalarString)

You can get rid of the quotes around the port by specifying it as an int.

import sys
import ruamel.yaml
from ruamel.yaml.comments import CommentedMap as omap
from ruamel.yaml.scalarstring import DoubleQuotedScalarString as dq
from ruamel.yaml.scalarstring import SingleQuotedScalarString as sq

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4)

device_name = "NN41_R11"
ip = "10.110.11.11"
port = 2022

def genyam():
    # initialise omap with list of tuples that are key-value-pairs
    data = omap([
        ('testbed', omap([('name', dq('boot_ios'))])),
    ])
    # or add in the order you want them in the YAML document
    data['devices'] = devices = omap()
    devices[device_name] = name = omap()
    name['type'] = 'IOS'
    name['connections'] = connections = omap()
    connections['defaults'] = omap([('class', sq('con.con')),])
    connections['a'] = a = omap()
    a['protocol'] = 'telnet'
    a['ip'] = ip
    a['port'] = port
    yaml.dump(data, sys.stdout)


genyam()

gives:

testbed:
    name: "boot_ios"
devices:
    NN41_R11:
        type: IOS
        connections:
            defaults:
                class: 'con.con'
            a:
                protocol: telnet
                ip: 10.110.11.11
                port: 2022

There is no way in ruamel.yaml (and even less so in PyYAML) to get different indents for different mappings as you have in your output (you have mostly four, but also five and two positions indent).

---

A completely different approach is to make a template for your YAML, and load and dump to make sure it is valid YAML (after filling out the template):

import sys
import ruamel.yaml

yaml_str = """\
testbed:
    name: "boot_ios"
devices:
    {device_name}:
        type: IOS
        connections:
            defaults:
                class: 'con.con'
            a:
                protocol: telnet
                ip: {ip}
                port: {port}
"""

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4)
yaml.preserve_quotes = True

def genyam2(device_name, ip, port):
    data = yaml.load(yaml_str.format(device_name=device_name, ip=ip, port=port))
    yaml.dump(data, sys.stdout)

genyam2(device_name = "NN41_R11", ip = "10.110.11.11", port = 2022)

This has the same output as the previous example, because on round-tripping order is preserved (and superfluous quotes as well if you specify yaml.preseve_quotes = True)