Search code examples
ansibleclouddevopscluster-computing

Assign a unique master IP to each replica IP with Ansible


I am trying to create configuration files for each replica IP in the inventory, where the number of configuration files matches the number of cores, and then assign a unique master IP from my master IP list to its corresponding replica's config file. This is what I have done so far:

- name: Create Cluster   
  hosts: primary_ips:replica_ips
  remote_user: "{{ ssh_user }}"
  become: true
  become_method: sudo
  vars:
    ssh_user: "ubuntu"
    gcc_ver: "12"

  tasks:
    - name: Install prereqs on system with apt
    apt:
      name:
        - make
        - pkg-config
        - build-essential
        - "gcc-{{ gcc_ver }}"
        - "g++-{{ gcc_ver }}"
        - zip
        - numactl
      update_cache: yes
      install_recommends: no
    when: ansible_pkg_mgr == "apt"

  - name: Set up Master IPs  
    run_once: true
    when: inventory_hostname in groups['primary_ips'] 
    set_fact:
      master_ips: "{{ groups['primary_ips'] }}"
    
  - name: Create cluster configuration files for the replica
    when: inventory_hostname in groups['replica_ips'] 
    template:
      src: replica.conf.j2
      dest: "/etc/cluster/600{{ item.1 }}/{{ item.1 }}.conf"
    with_nested: 
      - "{{ master_ips}}"
      - "{{ range(0, cores)|list }}"
    vars:
      port_number: "600{{ item.1 }}"
      master_ip: "{{ item.0 }}"

template config file is (replica.conf.j2):

port {{ port_number }}
replicaof {{ master_ip }}

and ansible_inventory is:

[primary_ips]
10.1.2.5
10.1.2.4

[replica_ips]
10.1.2.7
10.1.2.6

However in the above snipped code, the master_ip will be replaced by the last elements of master_ips for all replicas as for each replica_ip, it loops over all elements of master while I want something like this:

master_ips  = [10.1.2.4, 10.1.2.5]

replica_ips = [10.1.2.6, 10.1.2.7]

For replica ip = 10.1.2.6 create config files as follow:

0.conf
port 6000
replicaof 10.1.2.4

1.conf
port 6001
replicaof 10.1.2.4

2.conf
port 6002
replicaof 10.1.2.4

3.conf
port 6003
replicaof 10.1.2.4

For replica ip = 10.1.2.7 create config files as follow:

0.conf
port 6000
replicaof 10.1.2.5

1.conf
port 6001
replicaof 10.1.2.5

2.conf
port 6002
replicaof 10.1.2.5

3.conf
port 6003
replicaof 10.1.2.5

Solution

  • Given the inventory

    shell> cat hosts
    [primary_ips]
    10.1.2.5
    10.1.2.4
    
    [replica_ips]
    10.1.2.7
    10.1.2.6
    

    Create a dictionary of the IPs

      master_ips: "{{ dict(groups.replica_ips|zip(groups.primary_ips)) }}"
    

    gives

      master_ips:
        10.1.2.6: 10.1.2.4
        10.1.2.7: 10.1.2.5
    

    Then, in the loop, increment the port

        - template:
            src: replica.conf.j2
            dest: "/tmp/cluster/{{ item }}.conf"
          loop: "{{ range(cores) }}"
          vars:
            port_number: "{{ 6000 + item|int }}"
            master_ip: "{{ master_ips[inventory_hostname] }}"
          when: inventory_hostname in groups.replica_ips
    

    Example of a complete project

    shell> tree .
    .
    ├── ansible.cfg
    ├── hosts
    ├── pb.yml
    └── replica.conf.j2
    
    0 directories, 4 files
    
    shell> cat hosts
    [primary_ips]
    10.1.2.5
    10.1.2.4
    
    [replica_ips]
    10.1.2.7
    10.1.2.6
    
    shell> cat replica.conf.j2 
    port {{ port_number }}
    replicaof {{ master_ip }}
    
    shell> cat pb.yml 
    - hosts: primary_ips,replica_ips
    
      vars:
    
        cores: 4
        master_ips: "{{ dict(groups.replica_ips|zip(groups.primary_ips)) }}"
    
      tasks:
    
        - debug:
            var: master_ips
          run_once: true
    
        - file:
            state: directory
            dest: /tmp/cluster
          when: inventory_hostname in groups.replica_ips
    
        - template:
            src: replica.conf.j2
            dest: "/tmp/cluster/{{ item }}.conf"
          loop: "{{ range(cores) }}"
          vars:
            port_number: "{{ 6000 + item|int }}"
            master_ip: "{{ master_ips[inventory_hostname] }}"
          when: inventory_hostname in groups.replica_ips
    

    gives

    [email protected]:/tmp/cluster $ cat /tmp/cluster/0.conf
    port 6000
    replicaof 10.1.2.5
    
    [email protected]:/tmp/cluster $ cat /tmp/cluster/1.conf
    port 6001
    replicaof 10.1.2.5
    
    [email protected]:/tmp/cluster $ cat /tmp/cluster/2.conf
    port 6002
    replicaof 10.1.2.5
    
    [email protected]:/tmp/cluster $ cat /tmp/cluster/3.conf
    port 6003
    replicaof 10.1.2.5
    

    Ditto the results on 10.1.2.6