How to write a python script to remove duplicate ip address or duplicate subnet and remove the overlaps ip address or ip_subnet

Question

I have a list,say,

ip_related_list = ['192.168.1.1', '192.168.1.2', '192.168.1.0/24', '192.168.0.0/16', '10.1.1.1', '10.1.1.1', '10.1.1.1', '10.1.1.2','10.10.0.0/16','10.20.0.0/16','10.10.0.0/16'],

How to write a python script to remove duplicate ip address or duplicate subnet and remove the overlaps ip address or ip_subnet

The expected results should be :

192.168.0.0/16,
10.10.0.0/16, 
10.20.0.0/16,
10.1.1.1,
10.1.1.2

'192.168.1.1', '192.168.1.2' and '192.168.1.0/24' are within subnet '192.168.0.0/16'. So only the subnet is taken.

Answer 1

Use netaddr.cird_merge operation to summarize groups of IP subnets and addresses:

import netaddr

ip_related_list = ['192.168.1.1', '192.168.1.2', '192.168.1.0/24', '192.168.0.0/16', '10.1.1.1', '10.1.1.1', '10.1.1.1',
                   '10.1.1.2','10.10.0.0/16','10.20.0.0/16','10.10.0.0/16']
merged_subnets = netaddr.cidr_merge(ip_related_list)
print(merged_subnets)

---

[IPNetwork('10.1.1.1/32'), IPNetwork('10.1.1.2/32'), IPNetwork('10.10.0.0/16'), IPNetwork('10.20.0.0/16'), IPNetwork('192.168.0.0/16')]

To get string representation you can do:

merged_subnets = list(map(str, merged_subnets))

---

['10.1.1.1/32', '10.1.1.2/32', '10.10.0.0/16', '10.20.0.0/16', '192.168.0.0/16']