Is it possible to parse encrypted Ansible vault values from a dynamic inventory in a playbook?

Question

I have a dynamic inventory set up which pulls hosts and their variables from a MySQL database. The dynamic inventory itself is working perfectly.

Some of the variables inside the inventory are sensitive so I would prefer not to store them as plain text.

So as a test I encrypted a value using:

ansible-vault encrypt_string 'foobar'

Which resulted in:

!vault |
          $ANSIBLE_VAULT;1.1;AES256
          39653264643032353830336333356665663638353839356162386462303338363661333564633737
          3737356131303563626564376634313865613433346438630a343534363066393431633366393863
          30333636386536333166363533373239303864633830653566316663616432633336393936656233
          6633666134306530380a356664333834353265663563656162396435353030663833623166363466
          6436
Encryption successful

I decided to store the encrypted value as a variable inside MySQL. For the avoidance of doubt, after some testing, you can normalise the encrypted string to:

$ANSIBLE_VAULT;1.1;AES256
363635393063363466313636633166356562396562336633373239333630643032646637383866656463366137623232653531613135623464353932653665300a376461666332626538386263343732333039356663363132333533663339313466346435373064343931383536393736303731393834323964613063323362370a3361313132396239336130643839623939346438616363383932616639656463

You can test this format works with a simple playbook:

---
- hosts: all

  gather_facts: no
  
  vars:
    final_var: !vault "$ANSIBLE_VAULT;1.1;AES256\n363635393063363466313636633166356562396562336633373239333630643032646637383866656463366137623232653531613135623464353932653665300a376461666332626538386263343732333039356663363132333533663339313466346435373064343931383536393736303731393834323964613063323362370a3361313132396239336130643839623939346438616363383932616639656463"

  tasks:
    - name: Display variable
      debug:
        msg: "{{ final_var }}"

When the playbook is executed the following output is observed:

ok: [target] => {
    "msg": "foobar"
}

The difficulty arises when trying to get the variable (my_secret) from the inventory instead of referencing it directly in a file.

---
- hosts: all

  gather_facts: no
  
  vars:
    final_var: !vault "{{ my_secret }}"

  tasks:
    - name: Display variable
      debug:
        msg: "{{ final_var }}"

This results in:

fatal: [target]: FAILED! => {"msg": "input is not vault encrypted data"}

Now, while I've spoken a lot about the value being stored in the dynamic inventory in MySQL we can get a similar behaviour if we remove that from the equation.

---
- hosts: all

  gather_facts: no
  
  vars:
    secret: "$ANSIBLE_VAULT;1.1;AES256\n363635393063363466313636633166356562396562336633373239333630643032646637383866656463366137623232653531613135623464353932653665300a376461666332626538386263343732333039356663363132333533663339313466346435373064343931383536393736303731393834323964613063323362370a3361313132396239336130643839623939346438616363383932616639656463"
    final_var: !vault "{{ secret }}"

  tasks:
    - name: Display variable
      debug:
        msg: "{{ final_var }}"

This is now nearly identical to the working example but the encrypted string is not written inline but instead coming from another variable.

This results in the same error:

fatal: [target]: FAILED! => {"msg": "input is not vault encrypted data"}

This may indicate that the wider problem is that Ansible is for some reason unable to parse encrypted data stored as a variable. Perhaps when the YAML is being parsed it is literally trying to decrypt "{{ my_secret }}" rather than $ANSIBLE_VAULT;1.1;AES256 ....

Which kind of makes sense but I wanted to run that past you guys and ask if there is a way around this or if you recommend a different approach entirely. Thanks.

Answer 1

You'll be better off putting the variables into the encrypted files. Store the encrypted files in MySQL instead of encrypted variables. If you already "have a dynamic inventory set up which pulls hosts and their variables from a MySQL database" there shouldn't be a problem to modify the setup. Pull the encrypted files from the database and store them in host_vars (and/or group_vars, play vars, role vars ...) instead of storing encrypted variables in the inventory (and/or in the code of playbook, role, ...). This way, in the code, you don't care whether a variable is encrypted or not.

For example

shell> tree host_vars/
host_vars/
├── test_01
│   └── final_var.yml
├── test_02
│   └── final_var.yml
└── test_03
    └── final_var.yml

shell> cat host_vars/test_01/final_var.yml 
final_var: final_var for test_01

shell> cat host_vars/test_02/final_var.yml 
final_var: final_var for test_02

shell> cat host_vars/test_03/final_var.yml 
final_var: final_var for test_03

Encrypt the files. For example

shell> ansible-vault encrypt host_vars/test_01/final_var.yml
Encryption successful

shell> cat host_vars/test_01/final_var.yml
$ANSIBLE_VAULT;1.1;AES256
37363965336263366466336236336466323033353763656262633836323062626135613834396435
3665356363396132356131663336396138663962646434330a346433353039383864333638633462
35623034363338356362346133303262393233346439363264353036386337356236336135626434
6533333864623132330a346566656630376439643533373263303338313063373239343463333431
62353230323336383263376335613635616339383934313164323938363066616136373036326461
3538613937663530326364376335343438366139366639303230

Then the playbook below

- hosts: test_01,test_02,test_03
  tasks:
    - debug:
        msg: "{{ inventory_hostname }}: {{ final_var }}"

gives (abridged)

    "msg": "test_02: final_var for test_02"
    "msg": "test_01: final_var for test_01"
    "msg": "test_03: final_var for test_03"