Why am I not getting all the bytes of the image in the server?

I am building an iOS app that takes a photo and sends it to a TCP server running on my computer. The way I'm doing it is configuring the connection with Streams like this:

func setupCommunication() {
    var readStream: Unmanaged<CFReadStream>?
    var writeStream: Unmanaged<CFWriteStream>?

    CFStreamCreatePairWithSocketToHost(kCFAllocatorDefault,
        "192.168.1.40" as CFString, 2323, &readStream, &writeStream)

    outputStream = writeStream!.takeRetainedValue()
    outputStream.schedule(in: .current, forMode: .common)
    outputStream.open()
}

Then, when I press the camera button, the photo is taken and sent through the outputStream. Since the TCP server doesn't know how much data it has to read, the first 8 bytes correspond to the size of the image, and the image is sent right after, as we can see in this code:

func photoOutput(_ output: AVCapturePhotoOutput,
    didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {

    if let image = photo.fileDataRepresentation() {
        print(image)
        print(image.count)

        var nBytes = UInt64(image.count)
        let nData = Data(bytes: &nBytes, count: 8)
        _ = nData.withUnsafeBytes({
            outputStream.write($0, maxLength: nData.count)
        })

        _ = image.withUnsafeBytes({
            outputStream.write($0, maxLength: image.count)
        })

        outputStream.close()
    }
}

On the server side, which is written in C, I perform the next actions:

Read first 8 bytes to know the size of the image

printf("\n[*] New client connected\n");

while (n_recv < sizeof(uint64_t)) {
    if ((n = read(client_sd, buffer, BUF_SIZ)) == -1) {
        printf("\n[-] Error reading data from the client\n");
        close(client_sd);
        close(server_sd);
        return 0;
    }

    n_recv += n;
}

memcpy(&img_size, buffer, sizeof(uint64_t));

printf("\n[+] Client says he's going to send %llu bytes\n", img_size);

Allocate enough memory to store the received image, and if we already read any byte of the image next to the its size, copy it.

if ((img_data = (uint8_t *) malloc(img_size)) == NULL) {
    printf("\n[-] Error allocating memory for image\n");
    close(client_sd);
    close(server_sd);
    return 0;
}

n_recv -= sizeof(uint64_t);
if (n_recv > 0) {
    memcpy(img_data, buffer, n_recv);
}

From now on, n_recv is the number of bytes received of the image only, not including the first 8 bytes for the size. Then just read till the end.

while (n_recv < img_size) {
    if ((n = read(client_sd, buffer, BUF_SIZ)) == -1) {
        printf("\n[-] Error reading data from the client\n");
        close(client_sd);
        close(server_sd);
        return 0;
    }

    memcpy(img_data + n_recv, buffer, n);
    n_recv += n;
}

printf("\n[+] Data correctly recived from client\n");

close(client_sd);
close(server_sd);

This works pretty nice at the beginning. In fact, I can see that I'm getting the right number for the image size every time:

However, I'm not getting the full image, and the server just keeps waiting blocked in the read function. To see what's happening, I added this

printf("%llu\n", n_recv);

inside the loop for reading the image, to watch the number of bytes received. It stops in the middle of the image, for some reason I'm not able to explain:

What's the problem that is causing the communication to stop? Is the problem in the server code or is it something related to iOS app?

Solution

First, the C code looks okay to me.. but you realize you are missing return code/result handling in Swift?

In the C code you are checking the return value of recv to know if the bytes were read.. IE: You are checking if read returns -1..

However, in the swift code you make the assumption that ALL the data was written.. You never checked the result of the write operation on OutputStream which tells you how many bytes were written or returns -1 on failure..

You should be doing the same thing (after all, you did it in C).. For such cases I created two extensions:

extension InputStream {
    /**
     * Reads from the stream into a data buffer.
     * Returns the count of the amount of bytes read from the stream.
     * Returns -1 if reading fails or an error has occurred on the stream.
     **/
    func read(data: inout Data) -> Int {
        let bufferSize = 1024
        var totalBytesRead = 0

        while true {
            let buffer = UnsafeMutablePointer<UInt8>.allocate(capacity: bufferSize)
            let count = read(buffer, maxLength: bufferSize)
            if count == 0 {
                return totalBytesRead
            }

            if count == -1 {
                if let streamError = self.streamError {
                    debugPrint("Stream Error: \(String(describing: streamError))")
                }
                return -1
            }

            data.append(buffer, count: count)
            totalBytesRead += count
        }
        return totalBytesRead
    }
}

extension OutputStream {
    /**
     * Writes from a buffer into the stream.
     * Returns the count of the amount of bytes written to the stream.
     * Returns -1 if writing fails or an error has occurred on the stream.
     **/
    func write(data: Data) -> Int {
        var bytesRemaining = data.count
        var bytesWritten = 0

        while bytesRemaining > 0 {
            let count = data.withUnsafeBytes {
                self.write($0.advanced(by: bytesWritten), maxLength: bytesRemaining)
            }

            if count == 0 {
                return bytesWritten
            }

            if count < 0 {
                if let streamError = self.streamError {
                    debugPrint("Stream Error: \(String(describing: streamError))")
                }
                return -1
            }

            bytesRemaining -= count
            bytesWritten += count
        }

        return bytesWritten
    }
}

Usage:

var readStream: Unmanaged<CFReadStream>?
var writeStream: Unmanaged<CFWriteStream>?

//For testing I used 127.0.0.1
CFStreamCreatePairWithSocketToHost(kCFAllocatorDefault, "192.168.1.40" as CFString, 2323, &readStream, &writeStream)


//Actually not sure if these need to be retained or unretained might be fine..
//Again, not sure..
var inputStream = readStream!.takeRetainedValue() as InputStream
var outputStream = writeStream!.takeRetainedValue() as OutputStream

inputStream.schedule(in: .current, forMode: .common)
outputStream.schedule(in: .current, forMode: .common)

inputStream.open()
outputStream.open()


var dataToWrite = Data() //Your Image
var dataRead = Data(capacity: 256) //Server response -- Pre-Allocate something large enough that you "think" you might read..

outputStream.write(data: dataToWrite)
inputStream.read(data: &dataRead)

Now you get error handling (printing) and you have buffered reading/writing.. After all, you're not guaranteed that the socket or pipe or w/e.. the stream is attached to has read/written ALL your bytes at once.. hence the reading/writing chunks.