Asyncore client in thread makes the whole program crash when sending data immediately

Question

I write a simple program in python, with asyncore and threading. I want to implement a asynchorous client without blocking anything, like this:

How to handle asyncore within a class in python, without blocking anything?

Here is my code:

import socket, threading, time, asyncore
class Client(asyncore.dispatcher):
    def __init__(self, host, port):
        asyncore.dispatcher.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.connect((host, port))
mysocket = Client("",8888)
onethread = threading.Thread(target=asyncore.loop)
onethread.start()
# time.sleep(5)
mysocket.send("asfas\n")
input("End")

Now a exception will be throwed in send("asfas\n"), because I didn't open any server.

I think the exception in send function will call the handle_error function and won't affect the main program, but most of the time it crashes the whole program, and sometimes it works! And if I uncomment the time.sleep(5), it will only crash the thread. Why does it behave like this? Could I write a program that won't crash the whole program and don't use time.sleep() ? Thanks! Error message:

Traceback (most recent call last):
  File "thread.py", line 13, in <module>
    mysocket.send("asfas\n")
  File "/usr/lib/python2.7/asyncore.py", line 374, in send
    result = self.socket.send(data)
socket.error: [Errno 111] Connection refused

Answer 1

First of all, I would suggest not using the old asyncore module but to look into more modern and more efficient solutions: gevent, or going along the asyncio module (Python 3.4), which has been backported somehow to Python 2.

If you want to use asyncore, then you have to know:

• be careful when using sockets created in one thread (the main thread, in your case), and dispatched by another thread (managed by "onethread", in your case), sockets cannot be shared like this between threads it is not threadsafe objects by themselves

• for the same reason, you can't use the global map created by default in asyncore module, you have to create a map by thread

• when connecting to a server, connection may not be immediate you have to wait for it to be connected (hence your "sleep 5"). When using asyncore, "handle_write" is called when socket is ready to send data.

Here is a newer version of your code, hopefully it fixes those issues:

import socket, threading, time, asyncore

class Client(threading.Thread, asyncore.dispatcher):
    def __init__(self, host, port):
        threading.Thread.__init__(self)
        self.daemon = True
        self._thread_sockets = dict()
        asyncore.dispatcher.__init__(self, map=self._thread_sockets)

        self.host = host
        self.port = port
        self.output_buffer = []
        self.start()

    def run(self):
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.connect((self.host, self.port))
        asyncore.loop(map=self._thread_sockets)

    def send(self, data):
        self.output_buffer.append(data)

    def handle_write(self):
        all_data = "".join(self.output_buffer)
        bytes_sent = self.socket.send(all_data)
        remaining_data = all_data[bytes_sent:]
        self.output_buffer = [remaining_data]

mysocket = Client("",8888)
mysocket.send("asfas\n")

If you have only 1 socket by thread (i.e a dispatcher's map with size 1), there is no point using asyncore at all. Just use a normal, blocking socket in your threads. The benefit of async i/o comes with a lot of sockets.

EDIT: answer has been edited following comments.