Search code examples
pythonurllibhttplib

How to make httplib debugger infomation into logger debug level


By default httplib debug send, headers and reply information returns as logger.info,

Instead can how do i display send, headers and replay as part of Debug information?

import requests
import logging
import httplib
httplib.HTTPConnection.debuglevel = 1

logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

requests.get('http://httpbin.org/headers')

It prints

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP Connection (1):
httpbin.org
send: 'GET /headers HTTP/1.1\r\nHost: httpbin.org\r\nConnection: keep-alive\r\nA
ccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.8.
1\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: nginx
header: Date: Mon, 14 Dec 2015 12:50:44 GMT
header: Content-Type: application/json
header: Content-Length: 156
header: Connection: keep-alive
header: Access-Control-Allow-Origin: *
header: Access-Control-Allow-Credentials: true
DEBUG:requests.packages.urllib3.connectionpool:"GET /headers HTTP/1.1" 200 156
<Response [200]>

Solution

  • Thanks @Eli

    I could achieve using this post http://stefaanlippens.net/redirect_python_print

    import logging
    import sys
    import requests
    import httplib
    
    
    # HTTP stream handler
    class WritableObject:
        def __init__(self):
            self.content = []
        def write(self, string):
            self.content.append(string)
    
    # A writable object
    http_log = WritableObject() 
    # Redirection
    sys.stdout = http_log 
    
    # Enable 
    httplib.HTTPConnection.debuglevel = 2
    
    # get operation
    requests.get('http://httpbin.org/headers')
    
    # Remember to reset sys.stdout!
    sys.stdout = sys.__stdout__  
    debug_info = ''.join(http_log.content).replace('\\r', '').decode('string_escape').replace('\'', '')
    
    # Remove empty lines
    debug_info = "\n".join([ll.rstrip() for ll in debug_info.splitlines() if ll.strip()])
    

    It prints like

    C:\Users\vkosuri\Dropbox\robot\lab>python New-Redirect_Stdout.py
    send: GET /headers HTTP/1.1
    Host: httpbin.org
    Connection: keep-alive
    Accept-Encoding: gzip, deflate
    Accept: */*
    User-Agent: python-requests/2.8.1
    reply: HTTP/1.1 200 OK
    header: Server: nginx
    header: Date: Tue, 15 Dec 2015 09:36:36 GMT
    header: Content-Type: application/json
    header: Content-Length: 156
    header: Connection: keep-alive
    header: Access-Control-Allow-Origin: *
    header: Access-Control-Allow-Credentials: true
    

    Thanks

    Malli