C - Writing directly to an array in while loop while receiving data with recv

Question

Here is my code:

FILE *responseStorage = fopen("response", "w");

if(responseStorage == NULL){
    //error handling
}

int receivedTotal = 0;
while(1){
    ssize_t received = recv(sockfd, buf, BUFSIZ, 0);
    if( received == -1 ){
        //error handling
    }
    if( received == 0 ){
        //end of stream
        break;
    }
    receivedTotal += received;
    if(fwrite(buf, received, 1, responseStorage) != 1){
         //error handling
    }
    memset(buf, '\0', sizeof(buf));
}

fclose(responseStorage);

FILE *responseFile = fopen("response", "r");
char responseArray[receivedTotal];
if(fread(responseArray, receivedTotal, 1, responseFile) == 0){
   //error
}

I am calling a ssize_t received = recv(sockfd, buf, BUFSIZ, 0);, recieving data from server, saving how much data I received in receivedTotal += received; and writing that data to with fwrite(buf, received, 1, responseStorage) to my file FILE *responseStorage. After that at the end of the stream loop breaks and I open responseStorage file in r mode, make an array of the size receivedTotal, char responseArray[receivedTotal];, and with fread(responseArray, receivedTotal, 1, responseFile write that data from responeStorage file to responseArray.

Is there a way to write dirrectly to a responseArray? I have to validate response later on so I need it in an array. I know I would have to dynamically alocate the space for array with malloc. I want to avoid using receivedTotal and responseStorage.

Answer 1

You're already reading from your socket into buf, so all you have to do is write buf to a dynamically allocated string rather than responseStorage. Like you say you just have ti handle memory space to fit your response.

The inefficient but really easy way to do this is to reallocate storage every time you read. You can allocate storage for the sum of the previous response reads and the new string in buf, then write both strings to newly allocated space. You know the sum of the length of these strings +1 for the null byte, so you don't have to worry much about available allocated space. This is however pretty expensive because the reads get copied over and over again.

The slightly more complex way to do it would be to allocate one probably fairly large main response string buffer, keep track of its length total allocated space and use strncat to continue to concatenate buf until it's length exceeds the remaining space in response, +1 (for terminating nil byte). When there's not enough room, you can call realloc to obtain more memory. realloc is not very effficient (by C standards) because it is likely to need to allocate different space, copy existing data, and then return a new pointer.

if you wanted to be really clever, you could allocate one big buffer, and send read a pointer at the offset of the next available spot in buffer. You still might need to grow buffer but at least you don't need to copy it. buf then becomes your response array. This is the implementation I'll demonstrate:

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char**argv){
  size_t size = 2;
  size_t len  = 0;
  char* fullbuf = malloc(sizeof(char)*size);
  while(1) {
    ssize_t b = read(STDIN_FILENO, fullbuf + len, sizeof(char) * (size - len-1) );
    if( b < 0) {
      perror("Couldn't read from stdin: ");
      exit(2);
    } else if( b == 0 ){
      break;
    }
    if( b + len + 1 >= size) {
      // time to allocate more memory
      size = size * 2;
      fullbuf = realloc(fullbuf, sizeof(char) * size);
      if( fullbuf == NULL ){
        fprintf(stderr, "Couldn't allocate %zd more bytes of memory\n", size);
        exit(1);
      }
    }
    len += b;
  }
  fullbuf[len] = '\0'; //terminating null space
  printf("%s", fullbuf);
}

For this demonstration I read from stdin instead of a socket, but same idea. I read only as much data as is available in buf. When it's full (but the terminating byte), I double its space. Note that I set fullbuf to the output of realloc - it may be the same address, it may not.

To prove it works, I started at the rather insane 2 byte buffer, and double from there so there's lots of realloc calls. I grabbed 32k of lorem ipsum to use as input.

$ du -h file.txt
 32K    file.txt
$ shasum -a 256 file.txt
346f2adbd1fdca6bf3b03fb0a4d2fd0030e3363e9a9c5d1e22747e1bcc316e37  file.txt
$ ./t  < file.txt | shasum -a 256
346f2adbd1fdca6bf3b03fb0a4d2fd0030e3363e9a9c5d1e22747e1bcc316e37  -

Awesome, if the shasums are the same, that means I outputted file.txt exactly.