Get size of HTTP response in C

Question

I'm building a fuzzer (web application security tool) and I'm having some trouble building the proxy server. Below is my code for a function that takes in a char* http message, converts a host name in ascii to the IP address, and then establishes a connection with that host, forwarding the request. It then receives the response and sends it back to the client.

I've done this before in C++ using the Boost libraries but I really want control over what I'm doing, so I opted to write this in C. The problem I'm facing is that I'm not getting the entire response back so the content portion of the http request isn't getting sent back to the client. How large do I have to make the response string so that it can handle a request of any length? Is there a better way to do it than just setting the string size ridiculously large?

I did some research and it seems like using getaddrinfo is the way to go as far as making the requests but I'm not very familiar with it. I also get this error that says:

malloc: *** error for object 0x7fb8a8c04fe8: incorrect checksum for freed object - object was probably modified after being freed.

I originally thought it was because I wasn't making room for the null terminator on the string but it seems that isn't the issue.

Also if you see any other blatant bugs or problems, please feel free to point them out

int sock;
struct addrinfo hints, *res, *res0;
int error;
memset(&hints, 0, sizeof(hints));
hints.ai_family = PF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
error = getaddrinfo(hostname, "http", &hints, &res0);
if(error){
    puts("Error translating hostname");
    exit(1);
}
int reslen;
char* response = malloc(strlen(message)+1 * sizeof(char));
for(res = res0; res; res= res->ai_next){
    sock = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
    if(sock < 0) puts("sock is < 0");
    if(connect(sock, res->ai_addr, res->ai_addrlen) < 0){
        fprintf(stderr, "Error connecting to %s\n", hostname);
    }
    write(sock, message, len);
    reslen = read(sock, response, kMaxMess);
    response[reslen] = '\0';
}
close(sock);
return response;

Answer 1

Well, an HTTP request can realistically be of any size (e.g. huge numbers of headers or posted form data). If you really need it all in one big string, you will have to read it in chunks and realloc your buffer as necessary to contain it all.

Note, though, that HTTP requests can also contain binary data with embedded NUL characters. So, you really should return a length along with your buffer so that you can safely handle NULs.

Alternatively, if you just need to parse the response without storing it, you can consider incrementally parsing it while reading each chunk. That way you won't have to stick the whole response in memory.