Search code examples
pythoncurldnsperformance-testingpycurl

Getting DNS resolution time and response time with Python


Does PycURL or any other python pakcage provides information about :

  • lookup
  • connection time

I would like to get the same information as this cURL command does (without calling the command using subprocess):

Command

curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null http://stackoverflow.com/

Output:

Lookup time:    0.029
Connect time:   0.144
PreXfer time:   0.144
StartXfer time: 0.263

Total time: 0.803

Solution

  • Yes, PyCurl provides the information. You can derive the information using pycurl. It can give a lot of details, not just the ones you mentioned.

    Here's a sample code that you can use to derive the same information:

    import pycurl
    from io import BytesIO
    
    buffer = BytesIO()
    c = pycurl.Curl()
    c.setopt(c.URL, 'http://stackoverflow.com/')
    c.setopt(c.WRITEDATA, buffer)
    c.perform()
    body = buffer.getvalue()
    
    print('TOTAL_TIME: %f' % c.getinfo(c.TOTAL_TIME))
    print('CONNECT_TIME: %f' % c.getinfo(c.CONNECT_TIME))
    print('PRETRANSFER_TIME: %f' % c.getinfo(c.PRETRANSFER_TIME))
    print('STARTTRANSFER_TIME: %f' % c.getinfo(c.STARTTRANSFER_TIME))
    
    c.close()
    

    It gives the following results:

    TOTAL_TIME: 2.252639
    CONNECT_TIME: 0.331571
    PRETRANSFER_TIME: 0.331634
    STARTTRANSFER_TIME: 0.638206
    

    I found a GitHub link that mentions some of the other flags as well which can be used in your code. Here are the flags for a quick view:

    * EFFECTIVE_URL
    * HTTP_CODE
    * TOTAL_TIME
    * NAMELOOKUP_TIME
    * CONNECT_TIME
    * PRETRANSFER_TIME
    * REDIRECT_TIME
    * REDIRECT_COUNT
    * SIZE_UPLOAD
    * SIZE_DOWNLOAD
    * SPEED_UPLOAD
    * HEADER_SIZE
    * REQUEST_SIZE
    * CONTENT_LENGTH_DOWNLOAD
    * CONTENT_LENGTH_UPLOAD
    * CONTENT_TYPE
    * RESPONSE_CODE
    * SPEED_DOWNLOAD
    * SSL_VERIFYRESULT
    * INFO_FILETIME
    * STARTTRANSFER_TIME
    * REDIRECT_TIME
    * REDIRECT_COUNT
    * HTTP_CONNECTCODE
    * HTTPAUTH_AVAIL
    * PROXYAUTH_AVAIL
    * OS_ERRNO
    * NUM_CONNECTS
    * SSL_ENGINES
    * INFO_COOKIELIST
    * LASTSOCKET
    * FTP_ENTRY_PATH