Python serialize HTTPFlow object (MITM)

Question

I am using mitmproxy, a python man-in-the-middle (MITM) proxy for HTTP, to modify on the fly the HTTP request of a certain website.

Goal:

For test purposes, when it receives an HTTP request, it should save it (it receives a HTTPFlow object) and the next time the same request will be made i need to resent the exact same data/html/header/resourses/ecc.. to the browser.

Problem:

The obvious solution is to serialize the object but it isn't serializable!

I cannot simply keep it in memory because i need to restart the proxy during the tests

What can i do to achieve my goal?

Details:

I have already tried pickle, cPickle and marshal with the following errors:

• a class that defines __slots__ without defining __getstate__ cannot be pickled

• can't pickle CDataGCP objects

• ValueError: unmarshallable object

Ideas:

• 1) how much is it a bad idea to change the original object to make it serializable? and how can i do it?
• 2) what if the main process communicate with a second always-alive python process which simply keep the object in memory? are they still need to comunicate obj serializing them?

Answer 1

Found solution (the HTTPFlow obj has a method to get the state)

Imports

from mitmproxy.models import HTTPFlow
from mitmproxy.models import HTTPResponse

Save state:

cached_state = http_flow_response.response.get_state()

Load state:

# create the obj 
http_response = HTTPResponse(
        cached_state['http_version'], # "HTTP/1.1"
        cached_state['status_code'],  # 200
        cached_state['reason'],       # "Ok"
        cached_state['headers'],      # Headers(content_type="...")
        cached_state['content'])      # str
# send the obj
http_flow_request.reply(cached_http_response)