Twisted / Amp networking: Respond to all clients, not just one making a request

Question

I'm trying to learn my way around these newfangled "factory" style network libraries. Twisted comes with much acclaim, but is an absolute nightmare for me, since I am not familiar with lambda and thus I'm really not sure how to follow what the demo code is doing.

demo client:

from twisted.internet import reactor, defer
from twisted.internet.protocol import ClientCreator
from twisted.protocols import amp
from ampserver import Sum, Divide


def doMath():
    d1 = ClientCreator(reactor, amp.AMP).connectTCP(
        '127.0.0.1', 1234).addCallback(
            lambda p: p.callRemote(Sum, a=13, b=81)).addCallback(
                lambda result: result['total'])
    def trapZero(result):
        result.trap(ZeroDivisionError)
        print "Divided by zero: returning INF"
        return 1e1000
    d2 = ClientCreator(reactor, amp.AMP).connectTCP(
        '127.0.0.1', 1234).addCallback(
            lambda p: p.callRemote(Divide, numerator=1234,
                                   denominator=0)).addErrback(trapZero)
    def done(result):
        print 'Done with math:', result
    defer.DeferredList([d1, d2]).addCallback(done)

if __name__ == '__main__':
    doMath()
    reactor.run()

Demo Server:

from twisted.protocols import amp

class Sum(amp.Command):
    arguments = [('a', amp.Integer()),
                 ('b', amp.Integer())]
    response = [('total', amp.Integer())]


class Divide(amp.Command):
    arguments = [('numerator', amp.Integer()),
                 ('denominator', amp.Integer())]
    response = [('result', amp.Float())]
    errors = {ZeroDivisionError: 'ZERO_DIVISION'}


class Math(amp.AMP):
    def sum(self, a, b):
        total = a + b
        print 'Did a sum: %d + %d = %d' % (a, b, total)
        return {'total': total}
    Sum.responder(sum)

    def divide(self, numerator, denominator):
        result = float(numerator) / denominator
        print 'Divided: %d / %d = %f' % (numerator, denominator, result)
        return {'result': result}
    Divide.responder(divide)


def main():
    from twisted.internet import reactor
    from twisted.internet.protocol import Factory
    pf = Factory()
    pf.protocol = Math
    reactor.listenTCP(1234, pf)
    print 'started'
    reactor.run()

if __name__ == '__main__':
    main()

As I understand it, The client p.callRemote(Sum, a=13, b=81) and p.callRemote(Divide, numerator=1234, denominator=0) portions call Math.sum(13, 81) and Math.Divide(1234, 0), because the factory object's protocol is set to the Math class. Somehow when the client receives the return value from the Server, the sub-function Done(result) is called, which prints stuff to the screen.

This is great, but my comprehension sucks and every piece of documentation seems to expect this level of understanding already.

What I really want to be able to do is send data from a client to a server, and from a server to several clients that are connected. This method seems to forget about a client as soon as the exchange is over, and reactor.run() blocks on the client, preventing it from doing any other work.

People probably wish for this functionality every single day. How do I understand this?

Edit: I've tried to have a "check in" function for the clients to call, but this seems like it would drown my server in requests just to respond "Nothing to report". Also, it adds delay, in that clients only receive new data when they ask for it, instead of when it actually becomes available. The factory - reactor layout doesn't seem to expose the client info I need to store in order to respond to them arbitrarily.

Answer 1

You seem to have three questions:

1. "What does lambda mean in Python?"

   Which is covered by Python's documentation. If you still have a hard time reading code written this way though, you can use the fact that lambda x: y is just a shortcut to writing def my_function(x): return y. Anywhere you see a lambda, like for example, this:

   def foo(bar):
       return boz().addCallback(lambda result: qux(bar, result))
   

   you can always pull out the lambda into its own function so that it's easier for you to read, like this:

   def foo(bar):
       def callback_for_boz(result):
           return qux(bar, result)
       return boz().addCallback(callback_for_boz)
   

2. "How do I make input on one connection result in output on another?"

   Which is documented by Twisted's FAQ.

3. "How do I make Twisted talk to multiple clients / connect to multiple servers?"

   Which is also a twisted FAQ. The general idea here is that reactor.run() means "... and then run the whole program". The only reason you run any code before reactor.run() is to set up your initial timers, or listening sockets, or your first connection. You can call connectTCP or listenTCP as many times as you like, either before you run the reactor or in any callback that happens later in your program.