pyftpdlib slow .read on file blocks entire mainloop

Question

Helllo,

I am using a custom AbstractFS on pyftpdlib that maps files on a HTTP server to FTP. This files are returned by my implementation of open (of AbstractFS) which returns a httplib.HTTPResponse wrapped by the following class:

class HTTPConnWrapper:
    def __init__(self, obj, filename):
        # make it more file obj like
        self.obj = obj
        self.closed = True
        self.name = filename.split(os.sep)[-1]

    def seek(self, arg):
        pass

    def read(self, bytes):
        #print 'read', bytes
        read = self.obj.read(100) #we DONT read var byes, but 100 bytes
        #print 'ok'
        return read

The problem is that if a client is downloading files the entire server becommes sluggish. What can I do? Any ideas?

PS: And why just monkey patching everything with evenetlet does'nt magically makes everything work?

Answer 1

Ok, I posted a bug report on pyftpdlib:

I wouldn't even know what to recommend exactly as it's a problem which is hard to resolve and there's no easy or standard way to deal with it.

But I got a crazy solution to solve this problem without using pyftpdlib.

1. rewrite everything using wsgidav (which uses the cherrypy wsgiserver, so its threaded)
2. mount that WebDAV filesystem as native filesystem (net use on windows, mount.davfs on linux)
3. serve this mounted filesystem with any ftp server that can handle blocking file systems