Freeze on sending large file using QTcpSocket

Question

I'm trying to send and receive some large files (at least 16GB) from a PC to another. The application freeze when the client application received around 2GB and it consume almost all of my memory (also used up around 2GB of memory). I do not have this problems with the serverside.

Here is server code that send file

clock_t startTime = clock();
QTcpSocket *clientConnection = tcpServer->nextPendingConnection();
connect(clientConnection, SIGNAL(disconnected()),
            clientConnection, SLOT(deleteLater()));

QString sourceFileName("/path/to/source/sourcefile.big");
QByteArray baFName=sourceFileName.toLocal8Bit();
char* c_fileName=baFName.data();

char buf[BUFSIZ];
size_t size;
unsigned long long sendSize=0;
int source = open64(c_fileName, O_RDONLY, 0);
unsigned long long loopCount=0;
while ((size = read(source, buf, BUFSIZ)) > 0) {
    sendSize=clientConnection->write(buf, size);
    clientConnection->waitForBytesWritten();
    if(sendSize< size) {
       qWarning("transmit error!");
    }
    qDebug() << "Loop #" << ++loopCount << " send data: " << sendSize;

}

qDebug() << double( clock() - startTime ) / (double)CLOCKS_PER_SEC<< " seconds.";

clientConnection->disconnectFromHost();

The client side of the application already know how big the file it received and here are the code that received the file and write it to disk

clock_t startTime = clock();

QString sourceFileName("/path/to/target/targetfile.big");
unsigned long long targetSize=16447314864ULL;

unsigned long long loopCount=(targetSize / 8192ULL) + ( targetSize % 8192ULL > 0 ? 1 : 0);
QByteArray baFName=sourceFileName.toLocal8Bit();
char* c_fileName=baFName.data();

char buf[BUFSIZ];
size_t size;
unsigned long long sendSize=0;
int dest = open64(c_fileName, O_WRONLY | O_CREAT, 0644);

while (loopCount){
    if (tcpSocket->waitForReadyRead()){
        size=tcpSocket->read(buf, 8192);
        write(dest, buf, size);
        qDebug() << "Loop #" << loopCount << " receive data: " << size;
        loopCount--;
    }
}

qDebug() << double( clock() - startTime ) / (double)CLOCKS_PER_SEC<< " seconds.";

I'm using ubuntu 14.04 if it matter.

Answer 1

I've managed to resolved the freeze and memory hogging thing of the client side. It turns out that waitForReadyRead() is the cause of it.

I just remove it and check the output of tcpSocket->read. If it's zero, then i put a delay of several ms so that it is not becoming a processor hog this time.

I wonder if the serverside can be faster if i remove the waitForByteWritten(). I have tried to check the clientConnection->bytesToWrite() and limit it to 1GB, but it turns out the number isn't going down. Is there any other method that I could to use to replace waitForByteWritten() in synchronous way?