Chunking in gRPC with Python

Question

Can anyone share any Python server and client code that demonstrates the use of chunking in gRPC similar to https://jbrandhorst.com/post/grpc-binary-blob-stream/? Thanks.

Answer 1

Here is the gRPC Python version of Chunker. The main logic of chunking in the servicer is implemented using Python generator.

# server.py
_CHUNKER_SIZE = 4
_DATA_TO_SEND = 'Hello gRPC Python World!'

def _chunk_bytes(data, chunker_size):
    index = 0
    while index < len(data):
        yield chunker_pb2.Chunk(
            chunk=data[index:index+chunker_size]
        )
        index += chunker_size


class Chunker(chunker_pb2_grpc.ChunkerServicer):

    @staticmethod
    def Chunker(request, unused_context):
        return _chunk_bytes(
            _DATA_TO_SEND,
            _CHUNKER_SIZE)

The client-side is straightforward. It receives response and concatenates them.

with grpc.insecure_channel('localhost:50051') as channel:
    stub = chunker_pb2_grpc.ChunkerStub(channel)
    response_iterator = stub.Chunker(empty_pb2.Empty())
    received_bytes = bytes()
    for response in response_iterator:
        received_bytes += response.chunk
print('Concatenated Response:')
print(received_bytes)

Full version available in Gist: https://gist.github.com/lidizheng/825f1b255767a90fb3a5d4be54071678