IOCP server works with WebSocket connections. When the browser sends a close frame, the server deletes this client, the closesocket function is calling in client's object destructor. But even after the socket was closed, the GetQueuedCompletionStatus function continues to select events from this socket. Of course the result is false and 0 bytes transferred, but Client ptr and OVERLAPPED ptr are not NULL, and GetLastError returns 1236 (ERROR_CONNECTION_ABORTED)... so yeah, it is aborted, and the closesocket was called... But why it is still here??? And how to stop receiving this "useless" events? I can call continue in thead's loop, but it will waste CPU time if the function will select this removed client forever.
Here is the part of worker thread's loop:
while(WAIT_OBJECT_0 != WaitForSingleObject(EventShutdown, 0)){
DWORD BytesTransfered = 0;
OVERLAPPED *asyncinfo = nullptr;
client *Client = nullptr;
BOOL QCS = GetQueuedCompletionStatus(hIOCP, &BytesTransfered, (PULONG_PTR)&Client, &asyncinfo, INFINITE);
if(!Client ) break;
switch( QCS * (BytesTransfered > 0) * Client->OpCode() ){
case OP_TYPE_RECV:{
.....
switch( recv_buf[0] &0xFF ){
....
case FIN_CLOSE:
printf("FIN_CLOSE on client %u\n", Client->Socket());
default:{
RemoveClient(Client);
break;
}
}
}
case OP_TYPE_SEND:{
...
}
default:{
printf("Client %u (%lu bytes transferred, QCS is %d)\n", Client->Socket(), BytesTransfered, QCS);
break;
}
Client's destructor:
client::~client(){
while(!HasOverlappedIoCompleted(&asyncinfo)) Sleep(0);
closesocket(socket);
if( a_ctx ) delete a_ctx;
if( q_ctx ) delete q_ctx;
delete [] data_buffer;
printf("Client %u deleted\n", socket);
}
... and the server's log:
Client 296 from 127.0.0.1 (agent 1987)
Client 308 from 127.0.0.1 (supervisor)
Client 324 from 127.0.0.1 (supervisor)
TOTAL: 3 client(s)
Sending 33278 bytes to 324
Send finished for 324
Send finished for 308
Sending 40529 bytes to 324
Send finished for 324
Sending 41128 bytes to 324
Send finished for 324
Sending 40430 bytes to 324
Send finished for 324
FIN_CLOSE on client 324
Client 324 deleted
Client 324 (0 bytes transferred, QCS is 0)
Client 324 (0 bytes transferred, QCS is 0)
Client 324 (0 bytes transferred, QCS is 0)
Client 324 (0 bytes transferred, QCS is 0)
Client 324 (0 bytes transferred, QCS is 0)
See that "324 (0 bytes transferred, QCS is 0)"? Socket 324 closed. Why it happens after destructor's message "Client 324 deleted"?
I don't think you've included enough code to get a complete picture, but this line looks suspect:
switch( QCS * (BytesTransfered > 0) * Client->OpCode() ){
This is problematic for a couple of reasons. First, GetQueuedCompletionStatus is not guaranteed to return 1 on success. MSDN promises only that it will return nonzero. Relying on a specific value for the success case is therefore risky. Second, you have no way to distinguish between a failed call and a successful call that returns 0 bytes. You should really separate your logic for managing the dequeue and for dispatching particular I/O events. This will make your code easier to understand and maintain.
You must also remember that there are two sides to every socket. There are the structures and socket handle you associate with it in user space, and then there are kernel objects that manage the low level details. Just because you close your handle on the user side does not mean that the kernel objects go away. Kernel objects are referenced counted and will generally linger on until all I/O involving those objects is complete.
That's why you can still get I/O notifications for a socket after it has been "destroyed" from your program's point of view. With sockets in particular, the shut down sequence will take place after you have closed your handle (because you did not explicitly shut down the socket before then).
Instead of destroying your Client object in response to a specific message, just close the socket handle and clean up your other structures in response to the abort notification. You might also consider doing a graceful disconnect rather than aborting the connection.