Implement gethostbyaddr() with asyncore

Question

I was having fun with socket.gethostbyaddr(), searching how to speed up a really simple code that generate some IP address randomly and try to solve them. The problem comes when no host can be found, there is a timeout that can be really long (about 10 seconds...)

By chance, I found this article, he solves the problem by using Multi-threading : https://www.depier.re/attempts_to_speed_up_gethostbyaddr/

I was wondering if it is possible to do something equivalent using Asyncore ? That's what I tried to do first but failed miserably...

Here is a template :

import socket
import random

def get_ip():
    a = str(random.randint(140,150))
    b = str(random.randint(145,150))
    c = str(random.randint(145,150))

    for d in range(100):
        addr = a + "." + b + "." + c +"."+ str(1 + d) 
        yield addr

for addr in get_ip():               
    try:
        o = socket.gethostbyaddr(addr)
        print addr + "...Ok :"
        print "---->"+ str(o[0])
    except:
        print addr + "...Nothing"

Answer 1

You are looking for a way how to convert several IPs to names (or vice versa) in parallel. Basically it is a DNS request/response operation and the gethostbyaddr is doing this lookup synchronously, i.e. in a blocking manner. It sends the request, waits for the response, returns the result.

asyncio and similar libraries use so called coroutines and cooperative scheduling. Cooperative means that coroutines are written to support the concurency. A running coroutine explicitly returns the control (using await or yield from) to a waiting scheduler which then selects another coroutine and runs it until that one returns the control and so on. Only one coroutine can be running at a time. For a smooth run coroutines must not execute code for a longer time without returning the control. A blocking operation in a coroutine blocks the whole programs. That prohibits the usage of gethostbyaddr.

A solution requires support for asynchronous DNS lookups. A coroutine sends the DNS request, sets a timeout, arranges that a DNS response will be passed to it and returns the control. Thus multiple coroutines can send their requests one after another before they wait for all the responses.

There are third party libraries for async DNS, but I have never used them. Looking at aiodns examples, it seems quite easy to write the code you are looking for. asyncore.gather would be probably the core of such function.