Ansible playbook missing hosts in HTML report when using parallel execution (serial)

I have an Ansible playbook that runs on a large number of hosts (around 800) to generate an HTML report containing hostnames. The playbook is intended to run in parallel on all hosts.

When I set the serial parameter to a value like 500 or higher, the resulting HTML report contains fewer entries than expected (e.g., only 500 hosts). However, when I set serial: 1, I get all the entries in the report. While serial: 1 works, it's not a feasible solution as I want the playbook to run on all hosts concurrently.

Here's a simplified version of my playbook:

- name: Tail Logs and Generate Report
  hosts: all
  gather_facts: no
  ignore_unreachable: yes
  serial: 500  # The issue occurs with higher values

  tasks:
    - name: create header html
      lineinfile:
         path: /reports/daily_report.html
         line: "<!DOCTYPE html> <html> <head> <style>table{border-collapse:collapse;width:100%}td,th{border:1px solid #000;padding:8px;text-align:left}</style><title>Daily Report Details</title> </head> <body> <h1>Daily Report Details</h1> <table> <thead>
      <tr> 
      <th>PG Host Name</th>
      </tr> 
      </thead> <tbody>"
      create: yes
      delegate_to: localhost
      run_once: true
      ignore_errors: yes

    - name: Write content in report
      lineinfile:
      path: /reports/daily_report.html
      line: "<tr> 
          <td> {{ inventory_hostname }}</td>
          </tr>"
      insertafter: EOF
      delegate_to: localhost
      ignore_errors: yes
      no_log: true

    - name: create footer html
      lineinfile:
      path: /reports/daily_report.html
      line: "</tbody> </table> </body> </html>"
      delegate_to: localhost
      ignore_errors: yes
      run_once: true

I've tried debugging, checking host iteration logic, and reviewing error messages, but I'm unable to identify the root cause. The playbook seems to run fine in parallel with a lower serial value.

Could someone help me understand why increasing the serial value results in missing hosts in the HTML report? Are there any best practices or considerations when using high values for serial in Ansible playbooks?

Any guidance or suggestions would be greatly appreciated. Thank you!

Solution

Adding throttle: 1 helped to resolve the issue. This helps to run a particular task one by one for all servers and not simultaneously. Simultaneous execution on all servers was probably causing the problem. Please refer for more info.

- name: Tail Logs and Generate Report
  hosts: all
  gather_facts: no
  ignore_unreachable: yes

  tasks:
    - name: create header html
      lineinfile:
         path: /reports/daily_report.html
         line: "<!DOCTYPE html> <html> <head> <style>table{border-collapse:collapse;width:100%}td,th{border:1px solid #000;padding:8px;text-align:left}</style><title>Daily Report Details</title> </head> <body> <h1>Daily Report Details</h1> <table> <thead>
      <tr> 
      <th>PG Host Name</th>
      </tr> 
      </thead> <tbody>"
      create: yes
      delegate_to: localhost
      run_once: true
      ignore_errors: yes

    - name: Write content in report
      lineinfile:
      path: /reports/daily_report.html
      line: "<tr> 
          <td> {{ inventory_hostname }}</td>
          </tr>"
      insertafter: EOF
      delegate_to: localhost
      ignore_errors: yes
      no_log: true
      throttle: 1

    - name: create footer html
      lineinfile:
      path: /reports/daily_report.html
      line: "</tbody> </table> </body> </html>"
      delegate_to: localhost
      ignore_errors: yes
      run_once: true