Using twisted to process files

Question

I'm trying to set up a twisted xmlrpc server, which will accept files from a client, process them, and return a file and result dictionary back.

I've used python before, but never the twisted libraries. For my purposes security is a non issue, and the ssh protocol seems like overkill. It also has problems on the windows server, since termios is not available.

So all of my research points to xmlrpc being the best way to accomplish this. However, there are two methods of file transfer available. Using the xml binary data method, or the http request method.

Files can be up to a few hundred megs either way, so which method should I use? Sample code is appreciated, since I could find no documentation for file transfers over xml with twisted.

Update:

So it seems that serializing the file with xmlrpclib.Binary does not work for large files, or I'm using it wrong. Test code below:

from twisted.web import xmlrpc, server

class Example(xmlrpc.XMLRPC):
    """
    An example object to be published.
    """

    def xmlrpc_echo(self, x):
        """
        Return all passed args.
        """
        return x

    def xmlrpc_add(self, a, b):
        """
        Return sum of arguments.
        """
        return a + b

    def xmlrpc_fault(self):
        """
        Raise a Fault indicating that the procedure should not be used.
        """
        raise xmlrpc.Fault(123, "The fault procedure is faulty.")

    def xmlrpc_write(self, f, location):
        with open(location, 'wb') as fd:
            fd.write(f.data)

if __name__ == '__main__':
    from twisted.internet import reactor
    r = Example(allowNone=True)
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()

And the client code:

import xmlrpclib
s = xmlrpclib.Server('http://localhost:7080/')
with open('test.pdf', 'rb') as fd:
    f = xmlrpclib.Binary(fd.read())
s.write(f, 'output.pdf')

I get xmlrpclib.Fault: <Fault 8002: "Can't deserialize input: "> when I test this. Is it because the file is a pdf?

Answer 1

XML-RPC is a poor choice for file transfers. XML-RPC requires the file content to be encoded in a way that XML supports. This is expensive in both runtime costs and network resources. Instead, try just POSTing or PUTing the file using plain old HTTP.