How do I design a twisted factory to handle disconnects?

Question

I have a ReconnectingClientFactory in a module. I'd like the module to be as flexible as possible. I only need a single TCP connection. I use the factory as a persistent interface to this connection. In the past the factory would respond to disconnections by endlessly retrying the connection, never informing the top level script (the script that imports the module) that there were connection problems.

Here's a brief example of what I have:

Factory(protocol.ReconnectingClientFactory):

    def clientConnectionFailed(self, connector, reason):
        ...

    def clientConnectionLost(self, connector, reason):
        ...

I think it's best if I inform the top level script (the script that imports the module) when there are connection problems. This way the top level script can define disconnect resolution behavior rather than it all being hard coded in the module. But what is the best way to communicate the connection problems to the top level script?

I could raise an exception, but where would it be caught? I guess the reactor would catch it, but how does that help?

There are no callbacks or errbacks I can fire to inform the top script of the connection problem.

The top script could provide specific functions [as arguments] to be called when connection problems occur. Is this good design though?

Answer 1

This question is a bit too abstract to provide a direct answer. It depends on what your top-level module is doing.

However, you should consider using endpoints, rather than ClientFactory. This might address some of your design questions. Receiving connection lost notifications is a little tricker (since ClientFactory.clientConnectionLost is actually a duplicate notification of IProtocol.connectionLost, it no longer exists in the endpoints API; so you have to wrap the IProtocol object if you care about that) but it does let you use more generic mechanisms for retrying failed connections, as instead of clientConnectionFailed, you simply get an errback on the Deferred that you got back from connect. So, for example, if all you wanted to do was "keep reconnecting until you succeed", you could use this totally generic Deferred-retry-loop instead of something specific to connections like ReconnectingClientFactory:

# Warning, untested, sorry if it's broken.
@inlineCallbacks
def retry(deferredThing, delay=30.0, retryCount=5):
    retries = retryCount
    while True:
        try:
            result = yield deferredThing()
        except:
            if not retries:
                raise
            retries -= 1
            log.err()
            yield deferLater(reactor, delay, lambda : None)
        else:
            returnValue(result)

Similarly, if you could make that deferredThing function return a Deferred which would fire only when your protocol's application logic was complete, in addition to calling IStreamServerEndpoint.connect, watched for connectionLost and would fail if the connection was lost before the interesting logic was complete.

Deferreds can be an effective way to manage this kind of asynchronous re-try state across multiple levels of a system.