I have an Ansible playbook to perform the following task:
Access a remote host via ssh and create a text file that lists all the backups on the remote host filesystem that can be inserted into an email body. I am looking for a more efficient way to write this or to find a best practice to implement this.
Any and all feedback is greatly appreciated!
Below is the current iteration in use to locate backups on the remote host:
- name: Create backup log from Remote Host
shell: |
cd /mnt/bntestmnt; echo -e "Backup server: [ipv4 address goes here] \nBackup directory: /export
/Backups/ESG \n\nConfluence Backups:" > BackupStatus.txt
ls -td Confluence/daily/* | xargs du -sh >> BackupStatus.txt; echo " " >> BackupStatus.txt;
ls -td Confluence/weekly/* | xargs du -sh >> BackupStatus.txt; echo -e "\nGitlab Backups:" >>
BackupStatus.txt
ls -td GitLab/daily/* | xargs du -sh >> BackupStatus.txt; echo " " >> BackupStatus.txt
ls -td GitLab/weekly/* | xargs du -sh >> BackupStatus.txt; echo -e "\nJira Backups:" >>
BackupStatus.txt
ls -td Jira/daily/* | xargs du -sh >> BackupStatus.txt; echo " " >> BackupStatus.txt
ls -td Jira/weekly/* | xargs du -sh >> BackupStatus.txt; cat BackupStatus.txt
register: Backup_Status
delegate_to: [remote host name goes here]
Because '>>' redirects the command output to a text file, Ansible cannot see the result, so 'cat' is used to display the final product so it can be logged as a variable, to be inserted into the email body.
This is the expected output:
Backup server: [ipv4 address goes here]
Backup directory: /export/Backups/ESG
Confluence Backups:
4.1G Confluence/daily/Thu-23Mar23
4.0G Confluence/daily/Wed-22Mar23
4.0G Confluence/daily/Tue-21Mar23
4.0G Confluence/daily/Mon-20Mar23
4.0G Confluence/daily/Sun-19Mar23
3.9G Confluence/daily/Sat-18Mar23
3.9G Confluence/daily/Fri-17Mar23
3.7G Confluence/weekly/Fri-10Mar23
3.5G Confluence/weekly/Fri-03Mar23
3.4G Confluence/weekly/Fri-24Feb23
3.3G Confluence/weekly/Sat-18Feb23
Gitlab Backups:
2.5G GitLab/daily/Thu-23Mar23
2.5G GitLab/daily/Wed-22Mar23
2.4G GitLab/daily/Tue-21Mar23
2.4G GitLab/daily/Mon-20Mar23
2.4G GitLab/daily/Sun-19Mar23
2.3G GitLab/daily/Sat-18Mar23
2.3G GitLab/daily/Fri-17Mar23
2.2G GitLab/weekly/Fri-10Mar23
2.1G GitLab/weekly/Fri-03Mar23
2.2G GitLab/weekly/Fri-24Feb23
Jira Backups:
2.6G Jira/daily/Thu-23Mar23
2.6G Jira/daily/Wed-22Mar23
2.6G Jira/daily/Tue-21Mar23
2.6G Jira/daily/Mon-20Mar23
2.6G Jira/daily/Sun-19Mar23
2.6G Jira/daily/Sat-18Mar23
2.6G Jira/daily/Fri-17Mar23
2.6G Jira/weekly/Fri-10Mar23
2.6G Jira/weekly/Fri-03Mar23
2.6G Jira/weekly/Fri-24Feb23
2.6G Jira/weekly/Fri-17Feb23
The list within each application has a newline between the last instance of 'daily' and the first instance of 'weekly'
Some alternate formats I have tried to use are below. I ran into issues in trying to format the output to achieve the desired result.
`ls -td Confluence/{daily,weekly}/* | xargs du -sh`
or
`find Confluence -mindepth 2 -type d -exec du -sh {} \;`
(This version seems to sort the list incorrectly, as the size/dates are out of order)
I've also looked into modifying the task to utilize Jinja2 to create a template, but feel that I may be out of my depth with that method. See below for the attempted format:
- name: Create backup log from Remote Host
template:
src: backup_report.j2
dest: /mnt/bntestmnt/BackupStatus.txt
vars:
confluence_backups: [variable goes here]
gitlab_backups: [variable goes here]
jira_backups: [variable goes here]
I'm still trying to figure out how I would write the contents of "backup_report.j2", but I have the following inside:
Backup server: [ipv4 address]
Backup directory: /export/Backups/ESG
Confluence Backups:
{% for backup in confluence_backups %}
{{ backup }}
{% endfor %}
Gitlab Backups:
{% for backup in gitlab_backups %}
{{ backup }}
{% endfor %}
Jira Backups:
{% for backup in jira_backups %}
{{ backup }}
{% endfor %}
Given the simplified tree both on the host test_11 and test_13
shell> ssh admin@test_11 find /tmp/bntestmnt
/tmp/bntestmnt
/tmp/bntestmnt/Gitlab
/tmp/bntestmnt/Gitlab/weekly
/tmp/bntestmnt/Gitlab/weekly/Fri-10Mar23
/tmp/bntestmnt/Gitlab/weekly/Fri-03Mar23
/tmp/bntestmnt/Gitlab/daily
/tmp/bntestmnt/Gitlab/daily/Sat-18Mar23
/tmp/bntestmnt/Gitlab/daily/Sun-19Mar23
/tmp/bntestmnt/Gitlab/daily/Fri-17Mar23
/tmp/bntestmnt/Confluence
/tmp/bntestmnt/Confluence/weekly
/tmp/bntestmnt/Confluence/weekly/Fri-03Mar23
/tmp/bntestmnt/Confluence/weekly/Fri-10Mar23
/tmp/bntestmnt/Confluence/daily
/tmp/bntestmnt/Confluence/daily/Sat-18Mar23
/tmp/bntestmnt/Confluence/daily/Sun-19Mar23
/tmp/bntestmnt/Confluence/daily/Fri-17Mar23
/tmp/bntestmnt/Jira
/tmp/bntestmnt/Jira/weekly
/tmp/bntestmnt/Jira/weekly/Fri-10Mar23
/tmp/bntestmnt/Jira/weekly/Fri-03Mar23
/tmp/bntestmnt/Jira/daily
/tmp/bntestmnt/Jira/daily/Sun-19Mar23
/tmp/bntestmnt/Jira/daily/Sat-18Mar23
/tmp/bntestmnt/Jira/daily/Fri-17Mar23
The play
- hosts: all
vars:
bck_remote_dir: /tmp/bntestmnt
bck_dir: /export/Backups/ESG
bck:
Confluence: [daily, weekly]
Gitlab: [daily, weekly]
Jira: [daily, weekly]
tasks:
- shell:
cmd: "ls -td {{ item.0.key }}/{{ item.1 }}/* | xargs du -sh"
chdir: "{{ bck_remote_dir}}"
with_subelements:
- "{{ bck|dict2items }}"
- value
register: bck_status
- debug:
var: bck_status
when: debug|d(false)|bool
- copy:
dest: /tmp/BackupStatus.txt
content: |
Backup server: {{ ansible_host }}
Backup directory: {{ bck_dir }}
{% for i in bck_status.results %}
{{ i.item.0.key }} {{ i.item.1 }} Backups:
{% for j in i.stdout_lines %}
{{ j }}
{% endfor %}
{% endfor %}
gives (the same for test_13)
shell> ssh admin@test_11 cat /tmp/BackupStatus.txt
Backup server: 10.1.0.61
Backup directory: /export/Backups/ESG
Confluence daily Backups:
4.5K Confluence/daily/Sun-19Mar23
4.5K Confluence/daily/Sat-18Mar23
4.5K Confluence/daily/Fri-17Mar23
Confluence weekly Backups:
4.5K Confluence/weekly/Fri-10Mar23
4.5K Confluence/weekly/Fri-03Mar23
Gitlab daily Backups:
4.5K Gitlab/daily/Sun-19Mar23
4.5K Gitlab/daily/Sat-18Mar23
4.5K Gitlab/daily/Fri-17Mar23
Gitlab weekly Backups:
4.5K Gitlab/weekly/Fri-10Mar23
4.5K Gitlab/weekly/Fri-03Mar23
Jira daily Backups:
4.5K Jira/daily/Sun-19Mar23
4.5K Jira/daily/Sat-18Mar23
4.5K Jira/daily/Fri-17Mar23
Jira weekly Backups:
4.5K Jira/weekly/Fri-10Mar23
4.5K Jira/weekly/Fri-03Mar23
Example of a complete playbook to create the tree for testing
shell> cat pb-create.yml
- hosts: all
vars:
bck_remote_dir: /tmp/bntestmnt
bck_files:
daily: [Fri-17Mar23, Sat-18Mar23, Sun-19Mar23]
weekly: [Fri-03Mar23, Fri-10Mar23]
bck:
Confluence: [daily, weekly]
Gitlab: [daily, weekly]
Jira: [daily, weekly]
tasks:
- file:
state: directory
path: "{{ bck_remote_dir }}"
- file:
state: directory
path: "{{ bck_remote_dir }}/{{ item.0.key }}/{{ item.1 }}"
with_subelements:
- "{{ bck|dict2items }}"
- value
- shell: "for file in {{ files }}; do cp /etc/passwd $file; done"
with_subelements:
- "{{ bck|dict2items }}"
- value
vars:
dir: "{{ bck_remote_dir }}/{{ item.0.key }}/{{ item.1 }}"
files: "{{ [dir]|product(bck_files[item.1])|
map('join', '/')|
join(' ') }}"