Figuring out which response belongs to which request using proxpy

I am writing a plugin for proxpy (you don't need to be familiar with it to answer this question). This is basically an HTTP/HTTPS proxy. You extend it by implementing two functions, the arguments to which is the HTTP request and response respectively. Something like this:

method1(request):  
    #your implementation

method2(response):  
    #your implementation

I want to simply write the requests and responses to a file in the following format.

Request 1  
Response of request 1  
Request 2  
Response of request 2.  
#etc

However, the requests and responses might not necessarily in an order so they are written something like this:

Request 1  
Request 2  
Response of request 2  
Request 3  
Response of request 3  
Response of request 2

So, in essence, what I want to do is find out which request each response corresponds to.

Solution

You could try something like the following code which stores each request in a dictionary keyed by thread id, and then matches the response using the same thread id. This relies on the fact that proxpy uses a separate worker thread for each request and response pair (something that requires knowledge of the proxpy implementation!).

Note that exceptions raised outside of the plugin code will result in "orphaned" requests in the sent dictionary, although this might not be such a problem if thread ids are reused (as observed in practise).

Also, the output of this plugin will be to log paired requests and responses, but the order will be that of reception of the response, not necessarily in the order sent.

import threading

lock = threading.Lock()
sent = {}
req_seq = 0

def proxy_mangle_request(req):
    global req_seq
    lock.acquire()
    req_seq += 1
    thread_id = threading.current_thread().ident
#    print "thread id = {}".format(thread_id)
    req.seq = req_seq
    sent[thread_id] = req
    lock.release()

    return req

def proxy_mangle_response(res):
    thread_id = threading.current_thread().ident
#    print "Got response for thread_id {}".format(thread_id)
    print "Request: {}".format(sent[thread_id].seq)
    lock.acquire()
    del sent[thread_id]
    lock.release()

    print "Response: {}".format(res)
    return res