Recently I start a project to make a search engine, I need to check all active domains which are hosted in my country too(iran). It doesn't matter what are the TLDs, the only thing matter is that the websites should hosted in iran. My problem is that I cant seek all existence IPs in my country then check if it is active or not or even it host a website or not because it takes a lot of time and resources, Is there any way that I can have access to all existence website that are hosted in iran? and how can I have a calculate the domains to scrap ?
In addition, I'm sorry for writing mistakes in my question.
if you want to have a list of all Domains that are hosted in iran, you have to send a request to the IP address that are belongs to iran, which there is 5,260,800 IP that are belongs to iran. look at this link iran (islamic republic-of) IP address ranges, you should send request to entire of IP of the link above, for instance look at the code below:
r = requests.get('http://79.127.127.15')
print(r.url)
this two line will give you the domain of your IP if there is one, and if there isn't any website behind the IP address, it will return you the same IP address,so you have to check every 5,260,800 IP to get the domains. also you can you the code below to generate IP address:
def Int2IP(ipnum):
o1 = int(ipnum / 16777216) % 256
o2 = int(ipnum / 65536) % 256
o3 = int(ipnum / 256) % 256
o4 = int(ipnum) % 256
return '%(o1)s.%(o2)s.%(o3)s.%(o4)s' % locals()
start_ip = ipaddress.IPv4Address('2.144.0.0')
end_ip = ipaddress.IPv4Address('2.147.255.255')
for ip_int in range(int(start_ip), int(end_ip)):
# print(requests.get('http://{}'.format(Int2IP(ip_int))))
print(Int2IP(ip_int))