epoll + non-blocking socket slower than blocking + timeout?

Question

I have two versions of a simple socket server written in python. The first version uses epoll + nonblocking aproach and seems to be a way slower than the version of the server with blocking socket + timeout.

Non-blocking Server spawns 10 children and children do an accept on socket. In this case all children are getting EPOLLIN notification, but only one child can do an accept, all other children will get EAGAIN and this is ignored by "except-block".

--- server-nonblocking.py ---

import socket, time, os, select

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('127.0.0.1', 10000))
sock.listen(512)
sock.setblocking(False)


for _ in range(0,10):

    pid = os.fork()
    if pid == 0:    #in child
        poll = select.epoll()
        poll.register(sock.fileno(), select.EPOLLIN)

        while True:
            events = poll.poll(3)   # listening for events with 2 sec timeout
            for fileno, event in events:
                if event & select.EPOLLIN:  # there is data on socket available
                    print("EPOLLIN in PID: " + str(os.getpid()))
                    try:
                        clientsock, addr = sock.accept()
                        clientsock.close()
                        print("accepted and closed in PID: " + str(os.getpid()))
                    except:
                        pass


# we are in parent process, keep it live
while True:
    time.sleep(10)

The blocking server also spawns 10 children but uses timeout on listening socket instead, timeouts are intercepted by except block and ignored:

--- server-blocking.py ---

import socket, time, os, select

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('127.0.0.1', 10000))
sock.listen(512)
sock.settimeout(5)


for _ in range(0,10):

    pid = os.fork()
    if pid == 0:    #in child
        while True:
            try:
                clientsock, addr = sock.accept()
                clientsock.close()
                print("accepted and closed in PID: " + str(os.getpid()))
            except:
                pass


# we are in parent process, keep it live
while True:
    time.sleep(10)

and here is the client. It only does connect to server in the loop and closes the connection. After 20 secs. the loop will be interrupted.

--- client.py ---

import socket, time, select, sys

i = 1
td = time.time()
while  True:
    print("loop " + str(i) + ", time: " + str(time.time() - td))
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(('localhost',10000))
    s.setblocking(False)
    s.close()
    i += 1
    if time.time() - td >= 20:
        break

And here are the results of both servers:

blocking:
loop 137670, time: 19.99994468688965

non-blocking:
loop 94051, time: 19.10340452194214

Blocking server could handle a way more connections than the non-blocking server. When the client is using the non-blocking version, I can see some delays on the loop.

Can someone explain this behaviour? Why there are some delays in the loop on epoll + non-blocking?

Thanks!!!

Answer 1

epoll() is useful for (from the man page, epoll(2)): monitoring multiple file descriptors to see if I/O is possible on any of them.

you are using epoll() to monitor one file descriptor. this is adding a bunch of overhead in terms of context switches; each child has to call epoll_create(), epoll_ctl(), and epoll_wait(). and then! they all get woken up for every new connection. and then! most of them fail with the accept.

in the blocking version, probably only one child even gets woken up.