MemoryError: mitmproxy has crashed
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: MemoryError
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: mitmproxy has crashed!
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: Please lodge a bug report at: https://github.com/mitmproxy/mitmproxy
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: 192.168.50.117:60549: Traceback (most recent call last):
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: File "/usr/local/lib/python3.6/site-packages/mitmproxy/proxy/server.py", line 121, in handle
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: root_layer()
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: File "/usr/local/lib/python3.6/site-packages/mitmproxy/proxy/modes/transparent_proxy.py", line 19, in call
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: layer()
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: File "/usr/local/lib/python3.6/site-packages/mitmproxy/proxy/protocol/tls.py", line 285, in call
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: layer()
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: File "/usr/local/lib/python3.6/site-packages/mitmproxy/proxy/protocol/http1.py", line 83, in call
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: layer()
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: File "/usr/local/lib/python3.6/site-packages/mitmproxy/proxy/protocol/http.py", line 190, in call
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: if not self._process_flow(flow):
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: File "/usr/local/lib/python3.6/site-packages/mitmproxy/proxy/protocol/http.py", line 397, in _process_flow
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: self.read_response_body(f.request, f.response)
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: MemoryError
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: Exception in response:[Errno 12] Cannot allocate memory
Jul 16 13:02:40 cubedev-PowerEdge-R330 bash[21256]: message repeated 37 times: [ Exception in response:[Errno 12] Cannot allocate memory]
Steps to reproduce the behavior:
Issue observed when 2 clients are connected to mitmdump.The crash is observed within 10 minutes
System Information
mitmdump --version Mitmproxy: 5.1.1 Python: 3.6.0 OpenSSL: OpenSSL 1.1.1g 21 Apr 2020 Platform: Linux-4.15.0-45-generic-x86_64-with-debian-stretch-sid
mitmproxy processes all request/responses in-memory, hence if you do a large download mitmproxy requires at least the same amount of RAM.
You can configure mitmproxy to "stream" (directly pass through) large requests and response bodies using the option stream_large_bodies:
mitmproxy --set stream_large_bodies=10m
This streams all bodies that are larger than 10MB (AFAIR a streamed body will not be processed by any filter and is also not captured).
Additionally you should save the collected requests/response using the -w option.