D std.zlib stream compression with http

Question

I'm trying to add gzip compression to a HTTP server i wrote in D. here is the code that dose the gzip encoding.

if ((Info.modGzip) & (indexOf(client.getRequestHeaderFieldValue("Accept-Encoding"),"gzip") != -1)){
    writeln("gzip");
    auto gzip = new Compress(HeaderFormat.gzip);
    client.addToResponseHeader("Content-Encoding: gzip");
    client.sendHeader("200 ok");
    while (0 < (filestream.readBlock(readbuffer))){
        client.client.send(gzip.compress(readbuffer));
    }
    client.sendData(gzip.flush(Z_FINISH));
    delete gzip;
    } else {
        writeln("no gzip");
        client.sendHeader("200 ok");
        while (0 < (filestream.readBlock(readbuffer))){
        client.client.send(readbuffer);
    }
    delete filestream;
}

but when i test it Firefox, internet explorer and chrome says that the encoding or compression is bad. why? the data is compressed with gzip.

Answer 1

Your code isn't sending the appropriate headers. The compression portion is fine, but the stuff surrounding it has a few bugs that need to be fixed.

cross posting what I said on the D newsgroup: http://forum.dlang.org/post/[email protected]

I'm trying to add gzip compression to a HTTP server i wrote in D. here is the code that dose the gzip encoding.

I know zlib gzip works for http, I used it in my cgi.d

            if(gzipResponse && acceptsGzip && isAll) {
                    auto c = new Compress(HeaderFormat.gzip); // want gzip

                    auto data = c.compress(t);
                    data ~= c.flush();

                    t = data;
            }

But your http server is buggy in a lot of ways. It doesn't reply to curl and doesn't keep the connection open to issue manual requests.

Among the bugs I see looking at it quickly:

server.d getRequestHeaderFieldValue, you don't check if epos is -1. If it is, you should return null or something instead of trying to use it - the connection will hang because of an out-of-bounds array read killing the handler.

You also wrote:

if ((Info.modGzip) & (indexOf(client.getRequestHeaderFieldValue("Accept-Encoding"),"gzip") != -1)){

Notice the & instead of &&. That's in fspipedserver.d.

Finally, you write client.client.send... which never sent the headers back to the client, so it didn't know you were gzipping! Change that to client.sendData (and change sendData in server.d to take "in void[]" instead of "void[]") and then it sends the headers and seems to work by my eyeballing.