Google Bigquery Backend Error with streaming inserts but not display in console

Question

I am getting following error from Google bigquer while doing streaming inserts:

Error message   Signet::AuthorizationError: Unexpected status code: 500. Server message: { "error": "internal_failure" }

I can understand that there can few errors but the same doesn't get reflected on console as shown below: As you can see in above, image, there are no 500 errors but yet in real there were 10 500 internal_failure errors.

Can you tell me why these errors don't refect on console & how do I ensure they don't happen?

Answer 1

This looks like a failiure to get your authentication token. This failure would occur before the client code even attempts to call the bigquery api, so the console you are looking at is accurately representing traffic.

I suspect it is a failure on a request to https://accounts.google.com/o/oauth2/token. Perhaps monitoring on outgoing http requests could verify this? (For example, see Getting error 500 when trying to obtain access token from authorization code and Internal_failure while getting refreshtoken using code?)

Back to the BigQuery API: when it returns an http error code 500, the error string will be one of "backendError" or "internalError". (For the curious: "backendError" is usually retriable, while "internalError" is likely a permanent failure.)