I want to compress HTTP traffic via gzip content-encoding http header, both on requests and responses. I'm able to receive compressed data from the HTTP server and to decompress it, but if I try to compress data using gzip format and send this data to the HTTP server, I receive 400 Bad Request, in Apache log I can see:
AH01387: Zlib: Invalid header
I have the following code:
public class HttpClientTester extends StandaloneApplication {
public static byte[] deflate(byte[] input) {
Deflater deflater = new Deflater();
deflater.setLevel(9);
deflater.setInput(input);
deflater.finish();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int compressedSize = deflater.deflate(buffer);
outputStream.write(buffer, 0, compressedSize);
}
return outputStream.toByteArray();
}
public static byte[] compress(byte[] input) throws IOException {
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
GZIPOutputStream zipStream = new GZIPOutputStream(byteStream);
zipStream.write(input);
zipStream.flush();
zipStream.finish();
zipStream.close();
return byteStream.toByteArray();
}
public static String deflateReverse(byte[] input) throws DataFormatException, UnsupportedEncodingException {
Inflater inflater = new Inflater();
inflater.setInput(input);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
while (!inflater.finished()) {
int decompressedSize = inflater.inflate(buffer);
outputStream.write(buffer, 0, decompressedSize);
}
return new String(outputStream.toByteArray(), "UTF-8");
}
public static String decompress(byte[] input) throws IOException {
GZIPInputStream zippedInputStream = new GZIPInputStream(new ByteArrayInputStream(input));
return Tools.readStream(zippedInputStream, StandardCharsets.UTF_8);
}
@Override
public Void execute(StandaloneApplicationContext arg0) throws Exception {
Bundle sslProperties = new Bundle();
sslProperties.set(SSLProperties.SSL_VERSION, "TLSv1.2");
sslProperties.set(SSLProperties.SSL_TRUST_ALL_CERTS, true);
HTTPSHandler httpsHandler = new HTTPSHandler(null, arg0.getLoggingContext().getLoggingId(), sslProperties);
HttpClientV2 clientV2 = new HttpClientV2(null, arg0.getLoggingContext().getLoggingId(), arg0.getConfigurationContext().getParametersReader());
clientV2.setSSLHandler(httpsHandler);
HttpRequest httpRequest = HttpFactory.createHttpRequest(HttpConstants.HTTP_CONTENT_TYPE_TEXT_HTML);
httpRequest.setTransactionId(arg0.getLoggingContext().getLoggingId());
httpRequest.setHeaderField("Accept-encoding", "gzip");
httpRequest.setHeaderField("Content-encoding", "gzip");
httpRequest.setURL("https://myserver/myapp");
httpRequest.setMethod(HttpMethod.POST);
byte[] unzippedInputBody = "<MessageEnvelop><Type>ERROR</Type><Anomaly>ERROR WHILE PARSING XML REQUEST</Anomaly></MessageEnvelop>".getBytes("UTF-8");
byte[] zippedInputBody = compress(unzippedInputBody);
httpRequest.setBody(zippedInputBody);
httpRequest.setHeaderField("content-length", "" + zippedInputBody.length);
HttpResponse httpResponse = clientV2.doCommunication(httpRequest);
byte[] stream = httpResponse.getBodyStream();
String data = decompress(stream);
//@formatter:off
pause(
"Data length.........: " + stream.length + "\r\n"+
"Real data...........: " + data + "\r\n"+
"Response B64 data...: " + new Base64().encode(stream) + "\r\n"+
"Request B64 data....: " + new Base64().encode(zippedInputBody));
//@formatter:on
return null;
}
public static void main(String[] args) {
StandaloneApplicationContext.run(new HttpClientTester());
}
}
I have set Apache HTTP server with the following settings in my VirtualHost:
SetInputFilter DEFLATE
SetOutputFilter DEFLATE
The objective is to compress the whole traffic (request and response). Now I'm able to decompress data when received by Apache HTTP server. Currently the uncompressed data is the following string:
<MessageEnvelop><Type>ERROR</Type><Anomaly>ERROR WHILE PARSING XML REQUEST</Anomaly></MessageEnvelop>
GZIP compressed (B64) data, currently being used both for request and response, please note that data are being sent in binary format, the base64 format is used for comparing my zipped stream and the received one:
H4sIAAAAAAAAALPxTS0uTkxPdc0rS83JL7CzCaksSLVzDQryD7LRB7NtHPPycxNzKiGCCuEenj6uCgGOQcGefu4KEb4+CkGugaGuwSE2+jCFNvpohgIA/IH0c2UAAAA=
The current XML is received when an invalid XML request has been received by web application. Without GZIP compression on HTTP request I am able to receive it compressed and able to decompress it in my Java client. If I try to compress my request as well, I receive 400 bad request from HTTP server, without any chance to reach my web application.
Attached the messages exchanged, exported by Java client logs:
All the above is correct. Problem was in the HttpClient object used. It manipulated gzip data as string instead of byte arrays. After fixing the management of HTTP requests' body, no problems occurred anymore.
Thanks.
