Google Analytics Core reporting API v3 needs 4 minutes to respond

Question

We have been using the Google Analytics core reporting API v3 in our Laravel application for over 2 years now. We have done some optimizations over the years but our setup has been the same and has been working for quite some time now.

Earlier this month something strange started happening, very rarely a request would take around 4 minutes to complete. The first occurrence I could find was on the 6th of December.

At first I thought nothing of it and assumed we hit some kind of limit, however the problem persisted and keeps showing up more and more.

So the past couple of days I have been searching for the root of the problem and I haven't the slightest idea about why this is happening. There is no sign in the developers console of hitting limits.

I took sometime to find out how many responses where slow when I kept calling the API in rapid succession and 4 out of 14 calls took between 240.3 and 240.5 seconds. I've printed the results of the calls and have only called 1 specific view to see if it wasn't an issue with sending to much data in 1 request. The view did not have data for the day so all requests came back with 0 rows and 0's as total for all metrics. There were no error messages appended.

The developer console says otherwise, it says non of the requests take over 300ms in the past hour:

This is happening on our servers and local environment so I do not think it has something to do with nginx/internet issues.

I am pretty sure my code is fine and this does have something to do with limits but I have added it anyway. This method pulls all the views we have stored in our database and does 3 calls to the API to pull multiple metrics, this can't be done in 1 request because you can't not request over 10 metrics at a time.

public function getExternal()
{
    $this->logInfo("Retrieving reports");
    $views = $this->getLinkedViews();
    $i = 1;
    foreach ($views as $view) {
        try {
            $params= array('dimensions' => 'ga:campaign, ga:source, ga:medium, ga:date' );
            $start = microtime(true);
            $response = $this->analytics->data_ga->get(
                'ga:' . $view->datasource_id,
                $this->startDate,
                $this->endDate,
                'ga:users, ga:sessions, ga:bounceRate, ga:avgSessionDuration, ga:goal1Completions , ga:goal2Completions , ga:goal3Completions , ga:goal4Completions',
                $params
                );
            var_dump('call time: ' . (microtime(true) - $start));
            $data = $response->getRows();
            if ($data != null) {
                $this->rawData[$view->datasource_id. '.1'] = $data;
            }
            $response = $this->analytics->data_ga->get(
                'ga:' . $view->datasource_id,
                $this->startDate,
                $this->endDate,
                'ga:goal5Completions , ga:goal6Completions , ga:goal7Completions , ga:goal8Completions , ga:goal9Completions , ga:goal10Completions , ga:goal11Completions , ga:goal12Completions',
                $params
                );
            $data = $response->getRows();
            if ($data != null) {
                $this->rawData[$view->datasource_id. '.2'] = $data;
            }
            $response = $this->analytics->data_ga->get(
                'ga:' . $view->datasource_id,
                $this->startDate,
                $this->endDate,
                'ga:goal13Completions , ga:goal14Completions , ga:goal15Completions , ga:goal16Completions , ga:goal17Completions , ga:goal18Completions , ga:goal19Completions , ga:goal20Completions',
                $params
            );
            $data = $response->getRows();
            if ($data != null) {
                $this->rawData[$view->datasource_id. '.3'] = $data;
            }
            $this->logInfo($view->name . ' metrics retrieved'. "(" .$i . " - " . count($views). ")", $view->datasource_id);
            $i++;
        } catch (\Google_Service_Exception $gse) {
            dd($gse);
            $this->logError($gse->getMessage(), isset($view->datasource_id) ? $view->datasource_id : null, $this->formatErrorCode($gse->getCode()));
        } catch (\Throwable $t){
            dd($t);
            $this->logError($t, isset($view->datasource_id) ? $view->datasource_id: null, $this->formatErrorCode('000001'));
        }

Answer 1

I reached out to Google, there response was as followed:

The team is aware of this issue, and is still currently working on a fix for this. For now, I would recommend the following suggestions in order to alleviate this issue:

• Simplify query complexity via reducing their scope (e.g. dimensions, columns, date ranges, filters, etc); see this page for a general idea of what drives complexity
• Implement exponential back-off

EDIT:
Google has fixed this issue, we had to implement the above suggestions because it seems we where getting throttled way more often than before the bug occurred.