IOCP ReadFile always blocks until read completion

Question

This is an example source for reading files using iocp.

It should be returned immediately because it makes an asynchronous call when calling ReadFile, which seems to work synchronously.

What is the problem?

he test environment is visual studio 2017 enterprise, windwos 10, The windows sdk version is 10.0.17763.0.

#define INVALID_HANDLE_VALUE ((HANDLE)(LONG_PTR)-1)

const int BUFFERSIZE = 1024 * 1024 * 400;
BYTE ReadBuffer[BUFFERSIZE] = { 0 };

DWORD WINAPI WaitQueue(LPVOID lpParam)
{
    auto hIocp = (HANDLE)lpParam;

    // WAIT COMPLETION QUEUE
    DWORD numberOfBytes;
    ULONG_PTR val;

    LPOVERLAPPED ov = { 0 };

    for (;;)
    {
        BOOL bSuccess = GetQueuedCompletionStatus(hIocp, &numberOfBytes, (PULONG_PTR)&val, &ov, INFINITE);

        SYSTEMTIME dequeTime;
        GetSystemTime(&dequeTime);

        Sleep(1000);

        printf("dequeue time %dsec %dmilli", dequeTime.wSecond, dequeTime.wMilliseconds);
    }
}

DWORD WINAPI ReadFileThread(LPVOID lpParam)
{
    Sleep(3000);

    auto hIocp = (HANDLE)lpParam;

    // CREATE FILE HANDLE
    auto fileName = "e:\\test.msi";

    auto hFile = CreateFile(fileName,
        FILE_READ_DATA,
        FILE_SHARE_READ,
        0,
        OPEN_EXISTING,
        FILE_FLAG_OVERLAPPED,
        0);

    if (hFile == INVALID_HANDLE_VALUE)
    {
        std::wcout << L"create file fail - " << fileName << std::endl;
        return 0;
    }

    // REGIST FILE HANDLE TO IOCP
    if (hIocp != CreateIoCompletionPort(hFile, hIocp, 0, 2))
    {
        auto err = GetLastError();

        std::cout << "add file handle fail:" << err << " - file handle:" << hIocp << std::endl;

        CloseHandle(hFile);
        CloseHandle(hIocp);
        return 0;
    }

    // READ FILE
    OVERLAPPED ol = { 0 };

    SYSTEMTIME startTime;
    GetSystemTime(&startTime);

    if (FALSE == ReadFile(hFile, ReadBuffer, _countof(ReadBuffer), 0, &ol))
    {
        if (GetLastError() != ERROR_IO_PENDING)
        {
            printf("Terminal failure: Unable to read from file.\n GetLastError=%08x\n", GetLastError());
            CloseHandle(hFile);
            return -1;
        }
    }

    DWORD d;
    GetOverlappedResult(hFile, &ol, &d, true);

    SYSTEMTIME endTime;
    GetSystemTime(&endTime);

    printf("start time %dsec %dmilli", startTime.wSecond, startTime.wMilliseconds);
    printf("end time %dsec %dmilli", endTime.wSecond, endTime.wMilliseconds);
}

int main()
{
    // CREATE ICOP
    auto hIocp = CreateIoCompletionPort(INVALID_HANDLE_VALUE, nullptr, 0, 1);

    if (hIocp == NULL)
    {
        auto err = GetLastError();

        std::cout << "Create IOCP failed. error:" << err << std::endl;
        return 0;
    }

    // CREATE READ THREAD
    CreateThread(
        NULL,
        0,
        ReadFileThread,
        hIocp,
        0,
        nullptr
    );

    // CREATE WAIT DEQUEUE THREAD
    CreateThread(
        NULL,
        0,
        WaitQueue,
        hIocp,
        0,
        nullptr
    );

    while (true)
    {
    }

    return 0;
}

Answer 1

first of all the iocp here is absolute unrelated. you try check are I/O operation (read in your case) on asynchronous file handle return immediately or block. how is iocp here related ? bind iocp to file is only way get notification, when I/O is complete. but this is absolute not affect are I/O byself block or return immediately. we can use any notification way here. (apc, iocp or event). for your test purpose the most simply use event.

then let look like - how you test - are read block or return in your code ? you not test this at all. test this need after ReadFile return - are operation completed or not. the I/O operation (ReadFile) is completed asynchronous - if api call already return control to you, but OVERLAPPED(IO_STATUS_BLOCK ) yet not updated by system, which mean that I/O still not completed. are OVERLAPPED updated we can check direct (Internal member of the OVERLAPPED structure is not STATUS_PENDING) or by call GetOverlappedResult with bWait set to FALSE

If this parameter is FALSE and the operation is still pending, the function returns FALSE and the GetLastError function returns ERROR_IO_INCOMPLETE.

so we can say that ReadFile completed asynchronously if next 4 codition is true:

1. ReadFile return FALSE
2. GetLastError() return ERROR_IO_PENDING
3. GetOverlappedResult(.., FALSE) return FALSE
4. GetLastError() return ERROR_IO_INCOMPLETE

you not check this in self code. instead you wait until io operation is full complete via GetOverlappedResult(.., true) and take time of this. and what sense do this ?

real code for test:

void tt(PCWSTR FileName)
{
    HANDLE hFile = CreateFile(FileName, FILE_GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, 0);
    if (hFile != INVALID_HANDLE_VALUE)
    {
        OVERLAPPED ov {};
        if (ov.hEvent = CreateEvent(0, 0, 0, 0))
        {
            char buf[1024];
            if (ReadFile(hFile, buf, sizeof(buf), 0, &ov))
            {
                DbgPrint("sync(#1)\n");
            }
            else
            {
                switch (GetLastError())
                {
                case ERROR_IO_PENDING:
                    ULONG n;
                    if (GetOverlappedResult(hFile, &ov, &n, FALSE))
                    {
                        DbgPrint("sync(#2)\n");
                    }
                    else 
                    {
                        switch (GetLastError())
                        {
                        case ERROR_IO_INCOMPLETE:
                            DbgPrint("async\n");
                            if (!GetOverlappedResult(hFile, &ov, &n, TRUE))
                            {
                                __debugbreak();
                            }
                            break;
                        default: __debugbreak();
                        }
                    }
                    break;
                default: __debugbreak();
                }
            }
            CloseHandle(ov.hEvent);
        }
        CloseHandle(hFile);
    }
}

note that result (synchronous or not) complete of read depend from are file data in cache. if you call it for file, which was not readed before (so data not in cache) possible api print "async", but if you call this function again for the same file - you next time faster of all (almost 100%) will view "sync(#2)"