Can't correctly dump Ansible vault into yaml with Python

Question

I have a python dictionary with an Ansible vault as a value. I can't seem to be able to correctly dump it into a yaml output with the correct formatting. I'm using the ansible-vault package to generate the encrypted data as follows:

from ansible_vault import Vault
import yaml

vault = Vault('secretpassword')
data = "secretvalue"

To encrypt the data I do as follows:

password = (vault.dump(data))

Which generates a correct Ansible vault

$ANSIBLE_VAULT;1.1;AES256
36323830636261313833333932326661613063663361656362626432303232636463643030396366
3132623365633138613862333034626261336638613233650a653765386332666537613231626537
37666461653439616564343133656263613134643235363539396435616461626338393338616365
3339383030373532310a366662386665326132373535393930663737383136363735646361383066
65623033343262643138633839666237643735366465373932316233373339643835

To be able to use this in a vault I append the following to it:

password = "!vault |\n" + (vault.dump(data))

This is so that I am able to use it in a host_var. And afterwards I add it to a dictionary with some other values which in turn I will dump with yaml.

hostvar_dict = {
    "a": "1",
    "secret_item": password,
    "b": "2"
}

Everything looks fine until here, things go wrong when I try to output the above dict to a yaml output.

print(yaml.dump(hostvar_dict))

Gives me the following output:

a: '1'
b: '2'
secret_item: '!vault |

  $ANSIBLE_VAULT;1.1;AES256

  36353763313938663936303630306161393433633765353936656139363937373365376563623937

  3762633462623434393036316264646535316233346166660a396634386439656437343162613365

  34613661366163643333393163333335343632356330343939396133333665336566623037306432

  3539366466353030310a313936376361366366316338636161303564346633373237363463373966

  39353731323564393365633465303663373932613631353364626437633561643134

  '

I've looked at the answers from yaml.dump adding unwanted newlines in multiline strings but these didn't give the right output. What I'm looking to get is:

a: "1"
b: "2"
secret_item: !vault |
  $ANSIBLE_VAULT;1.1;AES256
  36353763313938663936303630306161393433633765353936656139363937373365376563623937
  3762633462623434393036316264646535316233346166660a396634386439656437343162613365
  34613661366163643333393163333335343632356330343939396133333665336566623037306432
  3539366466353030310a313936376361366366316338636161303564346633373237363463373966
  39353731323564393365633465303663373932613631353364626437633561643134

Options with yaml.dump I've tried are: default_style="|",default_flow_style=False.

Is there a way to correctly dump the ansible-vault value in a yaml file in the way I want it?

Answer 1

The !vault in your expected output YAML document is a tag. Tags start with an exclamation mark, and if you dump a string to YAML that starts with an exclacmation mark, that string needs to be quoted.

In a similar vein, the pipe (|) indicates you want a literal style scalar, and including that in your string, will not get you that.

I don't know if you can do this with PyYAML, but with ruamel.yaml you can do:

import sys
import ruamel.yaml

yaml = ruamel.yaml.YAML()

secret = ruamel.yaml.comments.TaggedScalar("""\
$ANSIBLE_VAULT;1.1;AES256
36353763313938663936303630306161393433633765353936656139363937373365376563623937
3762633462623434393036316264646535316233346166660a396634386439656437343162613365
34613661366163643333393163333335343632356330343939396133333665336566623037306432
3539366466353030310a313936376361366366316338636161303564346633373237363463373966
39353731323564393365633465303663373932613631353364626437633561643134
""", style='|', tag='!vault')

data = dict(a=1, b=2, secret_item=secret)
yaml.dump(data, sys.stdout)

which gives:

a: 1
b: 2
secret_item: !vault |
  $ANSIBLE_VAULT;1.1;AES256
  36353763313938663936303630306161393433633765353936656139363937373365376563623937
  3762633462623434393036316264646535316233346166660a396634386439656437343162613365
  34613661366163643333393163333335343632356330343939396133333665336566623037306432
  3539366466353030310a313936376361366366316338636161303564346633373237363463373966
  39353731323564393365633465303663373932613631353364626437633561643134

As written before, the best thing to do if is try and round-trip your expected output. Once it is clear that ruamel.yaml can preserve the layout, analyse the loaded data. In which case you'll see that the value for the key secret_item will be a TaggedScalar.