Search code examples
c++winsockwinsock2winsockets

Winsock RIO: RIOReceive returns immediately with no bytesTransferred


I'm having problems with getting winsock RIO working. It seems that every time I post a RIOReceive it returns immediately with 0 bytes transferred, and my peer can't get a message through.

After posting a RIOReceive, I wait on the RIODequeCompletion, which deques immediately with numResults = 1, but when I inspect the bytesTransferred of the RIORESULT struct, it's 0. This tells me that I'm not setting this thing up properly, but I can't find docs or examples that tell me what else I should be doing.

The internet seems to have very little on RIO. I've looked through MSDN, Len Holgate with TheServerFramework, this site, and two GitHub RIO servers.

RIOEchoServer and RIOServer_sm9 are on GitHub, but I can't post more than two links (this is my first question on this site).

This code is just to get things proven. It's currently not set to use the sendCQ, doesn't handle errors well, etc...

Here's the prep work:

void serverRequestThread() {

//init buffers 

//register big buffers
recvBufferMain = rio.RIORegisterBuffer(pRecvBufferMain, bufferSize);
sendBufferMain = rio.RIORegisterBuffer(pSendBufferMain, bufferSize);

if (recvBufferMain == RIO_INVALID_BUFFERID) {
    cout << "RIO_INVALID_BUFFERID" << endl;
}

if (sendBufferMain == RIO_INVALID_BUFFERID) {
    cout << "RIO_INVALID_BUFFERID" << endl;
}

//create recv buffer slice
recvBuffer1.BufferId = recvBufferMain;
recvBuffer1.Offset = 0;
recvBuffer1.Length = 10000;

//create send buffer slice
sendBuffer1.BufferId = sendBufferMain;
sendBuffer1.Offset = 0;
sendBuffer1.Length = 10000;

//completion queue
recvCQ = rio.RIOCreateCompletionQueue(CQsize, NULL);
sendCQ = rio.RIOCreateCompletionQueue(CQsize, NULL);

if (recvCQ == RIO_INVALID_CQ) {
    cout << "RIO_INVALID_CQ" << endl;
}

if (sendCQ == RIO_INVALID_CQ) {
    cout << "RIO_INVALID_CQ" << endl;
}

//start a loop for newly accept'd socket
while (recvCQ != RIO_INVALID_CQ && sendCQ != RIO_INVALID_CQ) {

    //get accept'd socket
    struct sockaddr_in saClient;
    int iClientSize = sizeof(saClient);
    acceptSocket = accept(listenSocket, (SOCKADDR*)&saClient, &iClientSize);

    if (acceptSocket == INVALID_SOCKET) {
        cout << "Invalid socket" << endl;
        printError();
    }

    //register request queue
    requestQueue = rio.RIOCreateRequestQueue(
        acceptSocket,       //socket
        10,                 //max RECVs on queue
        1,                  //max recv buffers, set to 1
        10,                 //max outstanding sends
        1,                  //max send buffers, set to 1
        recvCQ,             //recv queue
        recvCQ,             //send queue
        pOperationContext   //socket context
        );

    if (requestQueue == RIO_INVALID_RQ) {
        cout << "RIO_INVALID_RQ" << endl;
        printError();
    }

I now post a RIOReceive:

    //start a loop to repin recv buffer for socket
    while (acceptSocket != INVALID_SOCKET) {

        //pin a recv buffer to wait on data
        recvSuccess = rio.RIOReceive(
            requestQueue,           //socketQueue
            &recvBuffer1,           //buffer slice
            1,                      //set to 1
            RIO_MSG_WAITALL,                        //flags
            0);                     //requestContext

        if (recvSuccess == false) {
            cout << "RECV ERROR!!!!!!!!\n";
            printError();
        }

        //wait for recv to post in queue

        //std::this_thread::sleep_for(std::chrono::milliseconds(3000));

As soon as I call RIODequeCompletion, it returns 1:

        numResults = 0;
        while (numResults == 0) numResults = rio.RIODequeueCompletion(recvCQ, recvArray, 10);

        if (numResults == RIO_CORRUPT_CQ) {

            cout << "RIO_CORRUPT_CQ" << endl;

        } else if (numResults == 0) {

            cout << "no messages on queue\n";

        } else if (numResults > 0) {

but when I inspect the bytesTransferred of the RIORESULT, it's always 0:

            if (recvArray[0].BytesTransferred > 0) {

                //process results
                if (pRecvBufferMain[0] == 'G') {

                    //set respnose html
                    strcpy(pSendBufferMain, responseHTTP);

                    sendSuccess = rio.RIOSend(
                        requestQueue,   //socketQueue
                        &sendBuffer1,   //buffer slice
                        1,              //set to 1
                        0,              //flags
                        0);             //requestContext

                } else if (pRecvBufferMain[0] == 'P') {

                    //process post 


                } else {
                    //recv'd a bad message

                }

            } //end bytesTransferred if statement

            //reset everything and post another recv

        }//end response if statement

        std::this_thread::sleep_for(std::chrono::milliseconds(100));

    }//end while loop for recv'ing

    std::this_thread::sleep_for(std::chrono::milliseconds(100));

}//end while loop for accept'ing

}// end function

Like I said, I'm probably not using RIOReceive correctly, and/or I'm not setting the correct socket options that I need to (none right now).

I appreciate any help with this.


Solution

  • Try removing RIO_MSG_WAITALL. There may be a bug whereby you're only getting the close notification (bytes == 0) rather than getting a completion with the data in it. Either way it would be interesting to see if the code works without the flag.

    Do my example servers and tests work on your hardware?