I'm having trouble with browsermob-proxy and its har exporting feature. Some response bodies are not logged (the entire "text" field is missing)
My setup (using browsermob-proxy 2.1.4)
curl -X POST http://localhost:8080/proxy?port=9091
curl -X PUT "http://localhost:8080/proxy/9091/har?captureHeaders=true&captureCookies=true&captureContent=true"
Some responses are fine:
"response": {
"status": 201,
"statusText": "Created",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [{
"name": "Cache-Control",
"value": "max-age=0, no-cache, no-store"
}, {
"name": "Content-Type",
"value": "application/json"
}, {
"name": "Date",
"value": "Thu, 15 Feb 2018 13:07:39 GMT"
}, {
"name": "Location",
"value": ...
}, {
"name": "Pragma",
"value": "no-cache"
}, {
"name": "Render-Time",
"value": "8"
}, {
"name": "Server",
"value": "openresty"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=31536000; includeSubDomains"
}, {
"name": "transfer-encoding",
"value": "chunked"
}, {
"name": "Connection",
"value": "keep-alive"
}],
"content": {
"size": 8607,
"mimeType": "application/json",
"text": "{ <actual json body> }",
"comment": ""
}, ...
But some are not (maybe it is the special content/mime type? or maybe it is the gzip content encoding?)
"response": {
"status": 200,
"statusText": "OK",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [{
"name": "Cache-Control",
"value": "max-age=0, no-cache, no-store"
}, {
"name": "Content-Encoding",
"value": "gzip"
}, {
"name": "Content-Type",
"value": "application/some.custom.type-v1+json"
}, {
"name": "Date",
"value": "Thu, 15 Feb 2018 13:07:39 GMT"
}, {
"name": "Pragma",
"value": "no-cache"
}, {
"name": "Render-Time",
"value": "92"
}, {
"name": "Server",
"value": "openresty"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=31536000; includeSubDomains"
}, {
"name": "Vary",
"value": "Accept-Encoding"
}, {
"name": "Content-Length",
"value": "1978"
}, {
"name": "Connection",
"value": "keep-alive"
}],
"content": {
"size": 7429,
"mimeType": "application/some.custom.type-v1+json",
"comment": ""
< there's nothing else here!! >
},
"redirectURL": "",
"headersSize": 444,
"bodySize": 1978,
"comment": ""
}, ...
Oh yea, both requests are over https, using MITM.
I think I found the answer to my own question
In BrowserMobHttpClient.java I found the following:
private boolean hasTextualContent(String contentType) {
return contentType != null && contentType.startsWith("text/") ||
contentType.startsWith("application/x-javascript") ||
contentType.startsWith("application/javascript") ||
contentType.startsWith("application/json") ||
contentType.startsWith("application/xml") ||
contentType.startsWith("application/xhtml+xml");
}
Looks like I'll have go make a custom build of browsermob-proxy to make it work.