I'm trying to create a basic dynamic inventory script for ansible based on JSON output. I'm new to jq but I've hit an issue where the dynamic script on ansible v2.9.14 & 2.9.15 doesn't like the output, but if I send the output to a file and then run Ansible against the output in the file, ansible works.
This is what happens:
dynamic inventory script output:
{
"all": {
"hosts": {
"ip-172-31-39-30.eu-west-1.compute.internal": null,
"ip-172-31-44-224.eu-west-1.compute.internal": null,
"ip-172-31-42-6.eu-west-1.compute.internal": null,
"ip-172-31-32-68.eu-west-1.compute.internal": null,
}
}
}
Ansible run and error:
$ ansible -i ./dynamic1.sh all -m ping -u ubuntu
[WARNING]: * Failed to parse /home/ubuntu/dynamic1.sh with script plugin: failed to parse executable inventory script results from /home/ubuntu/dynamic1.sh:
Expecting property name enclosed in double quotes: line 8 column 5 (char 242)
[WARNING]: * Failed to parse /home/ubuntu/dynamic1.sh with ini plugin: /home/ubuntu/dynamic1.sh:2: Expected key=value host variable assignment, got: {
[WARNING]: Unable to parse /home/ubuntu/dynamic1.sh as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
Now, if I output the dynamic script to a file, then run ansible again, it works:
$ ./dynamic1.sh > output.json
$ cat output.json
{
"all": {
"hosts": {
"ip-172-31-39-30.eu-west-1.compute.internal": null,
"ip-172-31-44-224.eu-west-1.compute.internal": null,
"ip-172-31-42-6.eu-west-1.compute.internal": null,
"ip-172-31-32-68.eu-west-1.compute.internal": null,
}
}
}
$ ansible -i output.json all -m ping -u ubuntu
[DEPRECATION WARNING]: Distribution Ubuntu 16.04 on host ip-172-31-42-6.eu-west-1.compute.internal should use /usr/bin/python3, but is using /usr/bin/python for
backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation
warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ip-172-31-42-6.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
ip-172-31-39-30.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
ip-172-31-32-68.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
ip-172-31-44-224.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
So it works...
This is the contents of dynamic1.sh. I know there will be better ways to do this but I just need a list of servers based on a matching variable in the JSON output that ansible can use.
$ cat dynamic1.sh
#!/bin/bash
echo "{"
echo " \"all\": {"
echo " \"hosts\": {"
curl --silent -X GET https://url.com/api/servers -H "Authorization: Token $token" -H "Content-Type: text/json" -H "Accept:application/json" | jq -r '.Result.servers[] | select(.ansible_local.local.local_facts.instance_type | tostring | contains("t2.micro")) | (.ansible_fqdn+"\": null,")' | sed 's/^/"/g'
echo " }"
echo " }"
echo "}"
Can anyone give me any help on why ansible accepts the file but not the output of the script?
In contrary to the Ansible inventory format, the inventory plugin script.py expects the attribute hosts to be a list (e.g. hosts:[ host1, host2, host3 ]) not a dictionary (e.g. hosts:{ host, host2, host3 }).
Inventory plugin yaml.py works with dictionaries of hosts
The JSON (or YAML, because JSON is a subset of YAML) inventory works fine
shell> cat hosts.json
{
"all": {
"hosts": {
"ip-172-31-39-30.eu-west-1.compute.internal",
"ip-172-31-44-224.eu-west-1.compute.internal",
"ip-172-31-42-6.eu-west-1.compute.internal",
"ip-172-31-32-68.eu-west-1.compute.internal"
}
}
}
shell> ansible-inventory -i hosts.json --list -vvv
...
Parsed /scratch/tmp/hosts.json inventory source with yaml plugin
{
"_meta": {
"hostvars": {}
},
"all": {
"children": [
"ungrouped"
]
},
"ungrouped": {
"hosts": [
"ip-172-31-32-68.eu-west-1.compute.internal",
"ip-172-31-39-30.eu-west-1.compute.internal",
"ip-172-31-42-6.eu-west-1.compute.internal",
"ip-172-31-44-224.eu-west-1.compute.internal"
]
}
}
But, the same file provided by the script will fail
shell> cat hosts.sh
#!/bin/bash
cat hosts.json
shell> ansible-inventory -i hosts.sh --list -vvv
...
Parsed /scratch/tmp/hosts.sh inventory source with script plugin
[WARNING]: Failed to parse /scratch/tmp/hosts.sh with script plugin: You defined a group 'all' with bad data for the host list: {'hosts': {'ip-172-31-39-30.eu- west-1.compute.internal': None, 'ip-172-31-44-224.eu-west-1.compute.internal': None, 'ip-172-31-42-6.eu-west-1.compute.internal': None, 'ip-172-31-32-68.eu- west-1.compute.internal': None}} ...
{
"_meta": {
"hostvars": {}
},
"all": {
"children": [
"ungrouped"
]
}
}
Inventory plugin script.py works with lists of hosts
The inventory plugin script.py works as expected when the attribute hosts is a list
shell> cat hosts.json
{
"all": {
"hosts": [
"ip-172-31-39-30.eu-west-1.compute.internal",
"ip-172-31-44-224.eu-west-1.compute.internal",
"ip-172-31-42-6.eu-west-1.compute.internal",
"ip-172-31-32-68.eu-west-1.compute.internal"
]
}
}
shell> ansible-inventory -i hosts.sh --list -vvv
...
Parsed /scratch/tmp/hosts.sh inventory source with script plugin
{
"_meta": {
...
},
"all": {
"children": [
"ungrouped"
]
},
"ungrouped": {
"hosts": [
"ip-172-31-32-68.eu-west-1.compute.internal",
"ip-172-31-39-30.eu-west-1.compute.internal",
"ip-172-31-42-6.eu-west-1.compute.internal",
"ip-172-31-44-224.eu-west-1.compute.internal"
]
}
}
Notes
description: - The source provided must be an executable that returns Ansible inventory JSON - The source must accept C(--list) and C(--host ) as arguments. C(--host) will only be used if no C(_meta) key is present.