python grpc: set timeout per one attempt

Question

I wrote something like this:

settings = {
    'methodConfig': [
        {
            'name': [{}],
            'timeout': '0.5s',
            'retryPolicy': {
                'maxAttempts': 5,
                'initialBackoff': '0.1s',
                'maxBackoff': '2s',
                'backoffMultiplier': 2,
                'retryableStatusCodes': [
                    'UNAVAILABLE',
                    'INTERNAL',
                    'DEADLINE_EXCEEDED',
                ],
            },
        },
    ],
}
settings_as_json_string = json.dumps(settings)
request = Request(...)
async with grpc.aio.insecure_channel(
   host_port, options=(('grpc.service_config', settings),),
) as channel:
    stub = StubClass(channel=channel)
    await stub.SomeMethod(
        request=request,
    )

On the service side, I specifically slowed down call processing so that it takes about one second. When running the above code, I see that there are no 5 call attempts with a 0.5 second wait each time. There is only one attempt, ending with error DEADLINE_EXCEEDED after about 0.5 seconds. From here I conclude that timeout is not the waiting time for the service response in each attempt, but it is the maximum duration of interaction with the service. A similar behavior is observed when I remove timeout from the config and specify it as argument in the call SomeMethod:

    ...
    await stub.SomeMethod(
        request=request,
        timeout=0.5,
    )

My question is this:

Is there a way to specify the timeout of waiting for the service response in each individual attempt? So after 0.5 seconds of waiting and some backoff, there was another attempt, during which we also waited for a response within 0.5 seconds, and so on until the max_attempts was reached.

Answer 1

There is no per-attempt timeout in the design of the retry functionality in gRPC. The general theory is that any attempt could succeed, and artificially cutting an attempt off early only reduces the chances of achieving any success at all.