Python - Twisted client - Maximum size of data in dataRecived method

Question

I've recently been using and learning twisted for creating TCP client socket.

I got that dataRecived method of Protocol class, returns all data together. It means all data ConCat with each other and finally it returns a long byte data from server.

Code:

from twisted.internet.protocol import Protocol

class ClientProtocol(Protocol):
    def connectionMade(self):
        self.transport.write(b'a')

    def dataReceived(self, data):
        print ('data', data)

So now my questions are:

1. Has dataRecived method any maximum size for data? Or it's size is unlimited?

2. If it has maximum size, what is that? And how can i override that?

Note: I'm using Ubuntu 14.04, Python v3.4 and Twisted v15.3.0.

Answer 1

According to Twisted Documentation , the data parameter in dataRecived method, is not unlimited and it is :

" a string of indeterminate length. Please keep in mind that you will probably need to buffer some data, as partial (or multiple) protocol messages may be received! " .

So we need to override dataRecived method manually for buffering data. Or using some defined inherit class from protcol.Protocol like: LineReceiver class or NetstringReceiver class for this issue.

For example dataReceived source of LineRecevier class is as bellow:

 def dataReceived(self, data):
        """Protocol.dataReceived.
        Translates bytes into lines, and calls lineReceived (or
        rawDataReceived, depending on mode.)
        """
        self.__buffer = self.__buffer+data
        while self.line_mode and not self.paused:
            try:
                line, self.__buffer = self.__buffer.split(self.delimiter, 1)
            except ValueError:
                if len(self.__buffer) > self.MAX_LENGTH:
                    line, self.__buffer = self.__buffer, ''
                    return self.lineLengthExceeded(line)
                break
            else:
                linelength = len(line)
                if linelength > self.MAX_LENGTH:
                    exceeded = line + self.__buffer
                    self.__buffer = ''
                    return self.lineLengthExceeded(exceeded)
                why = self.lineReceived(line)
                if why or self.transport and self.transport.disconnecting:
                    return why
        else:
            if not self.paused:
                data=self.__buffer
                self.__buffer=''
                if data:
                    return self.rawDataReceived(data)