I have a hyperledger fabric network deployed in GCP. I use fabric node sdk to communicate to the network.
When there's no activity between sdk and network, If i try to invoke the chaincode, the API call fails at sendTransaction to the Orderer with below logs.
error: [Orderer.js]: sendBroadcast - on error: "Error: 14 UNAVAILABLE: TCP Read failed\n at createStatusError (/usr/src/app/node_modules/grpc/src/client.js:64:15)\n at ClientDuplexStream._emitStatusIfDone (/usr/src/app/node_modules/grpc/src/client.js:270:19)\n at ClientDuplexStream._receiveStatus (/usr/src/app/node_modules/grpc/src/client.js:248:8)\n at /usr/src/app/node_modules/grpc/src/client.js:804:12" Request failed: POST /channels/stanfinandcredth/chaincodes/sc1547746253: SERVICE_UNAVAILABLE Error: SERVICE_UNAVAILABLE at ClientDuplexStream. (/usr/src/app/node_modules/fabric-client/lib/Orderer.js:136:21) at emitOne (events.js:116:13) at ClientDuplexStream.emit (events.js:211:7) at ClientDuplexStream._emitStatusIfDone (/usr/src/app/node_modules/grpc/src/client.js:271:12) at ClientDuplexStream._receiveStatus (/usr/src/app/node_modules/grpc/src/client.js:248:8) at /usr/src/app/node_modules/grpc/src/client.js:804:12 Error: SERVICE_UNAVAILABLE at ClientDuplexStream. (/usr/src/app/node_modules/fabric-client/lib/Orderer.js:136:21) at emitOne (events.js:116:13) at ClientDuplexStream.emit (events.js:211:7) at ClientDuplexStream._emitStatusIfDone (/usr/src/app/node_modules/grpc/src/client.js:271:12) at ClientDuplexStream._receiveStatus (/usr/src/app/node_modules/grpc/src/client.js:248:8) at /usr/src/app/node_modules/grpc/src/client.js:804:12
It seems that SDK is not able to reach to Orderer. The immediate next invoke request succeeds. this happens eerytime after idle time of 15 mins.
I tried setting grpc timeouts for the connection but it doesn't seem to help.
It's likely that idle connections are being torn down by GCP. You'll need to set the gRPC keepalive options for your client. Something like:
"connection-options": {
"grpc.max_receive_message_length": -1,
"grpc.max_send_message_length": -1,
"grpc.keepalive_time_ms": 120000,
"grpc.http2.min_time_between_pings_ms": 120000,
"grpc.keepalive_timeout_ms": 20000,
"grpc.http2.max_pings_without_data": 0,
"grpc.keepalive_permit_without_calls": 1
}
See https://fabric-sdk-node.github.io/release-1.4/tutorial-grpc-settings.html for details on setting these using the Node SDK