I have a grpc async client implementation that receives data from the server via a stream.
After setting up a grpc::Channel to the server, I periodically query the channel state via grpc::Channel::GetState(false). If the state is not GRPC_CHANNEL_READY I determine the connection to the server is lost.
I have a suspicion that grpc::Channel::GetState does not always work reliably (I am witnessing a lot of lost packets during Server streaming).
My thought was to try out grpc keepalive mechanism. However, beside how to set it up, I did not understand from the documentation how will it affect existing code behavior. I mean - what does it do exactly? Will grpc::Channel::GetState behave differently? How can I query the status of the keepalive ping pong?
With enabling the keepalive, the GetState will not behave differently.
What keepalive does is when you have an unfinished RPC call it will send an HTTP2 PING frame every GRPC_ARG_KEEPALIVE_TIME_MS and wait for an ACK PING frame for GRPC_ARG_KEEPALIVE_TIMEOUT_MS. If no ack ping is received within this time it will close the connection and all RPC calls on that connection will fail (and may be retried). When a retry (or any new call) is issued, a connection will be re-established (unless another existing connection is picked). So you don't need to query the status of keep alive ping pong.
GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS enables ping even when you don't have any calls in-flight, but server may not allow that. Make sure that client side keepalive configuration is in sync with the server side. Otherwise, server may initiate closing connection after getting some amount of unexpected pings.
Please note that keepalive is hop by hop, so if you have an http2 proxy it will only test connectivity to the proxy and keepalive must be configured on the proxy too to test the next hop.