Search code examples
node.jsgoogle-analyticsgoogle-apigoogle-analytics-apiquota

Google API dailyLimitExceeded solutions

I've been working with Google Analytics for 2 months now. I created a custom dashboard with NodeJS (express/serverless), out of it with requesting data from the Core Reporting API and the Real Time Reporting API. I've managed to put it as a Lambda Function on AWS. While I'm very pleased about this, I have some issues I'm facing right now.

I get the following errors:

{  
   "error":{  
      "errors":[  
         {  
            "domain":"global",
            "reason":"dailyLimitExceeded",
            "message":"Quota Error: profileId ga:NNNNN has exceeded the daily request limit."
         }
      ],
      "code":403,
      "message":"Quota Error: profileId ga:NNNNN has exceeded the daily request limit."
   }
}

and

{  
   "error":{  
      "errors":[  
         {  
            "domain":"usageLimits",
            "reason":"userRateLimitExceeded",
            "message":"User Rate Limit Exceeded"
         }
      ],
      "code":403,
      "message":"User Rate Limit Exceeded"
   }
}

My dashboard looks like this:

enter image description here

When the dashboard get visited, it calls the realtime api 9 times (each block in the image is a querycall). I think I could combine the 'Online users', 'Users today' and 'pageviews today' call into one call. The search today, and orders today are specified by filters for searching for a specific event.

I have a build in a timechecker, which allows the dashboard to be viewed between 07:00 and 19:00. When it's earlier then 07:00 or later then 19:00 a variable checkTime is set to false, which makes the dashboard shows a div with text something like "dashboard offline". When someone has visited the dashboard in the allowed timerange, a variable checkTime is set to true and calls to the Google API's can be made.

The dashboard is running on a tv screen between 07:00 and 19:00. This means that the dashboard is up on a TV screen for 12 hours long. Every 20 seconds there is a function call to update all the data (so again 9 requests are being made).

So let's say there are

60 minutes x 3 = 180 x 12 = 2160 x 9(requests) = 19440 requests for a day.

I don't think I should reach the 50.000 quota. But I am reaching the profile quota from 10.000.

However when I view the Developers console I can see the following:

enter image description here

enter image description here

enter image description here

I think my options are the following:

  • Increase the interval to 1 minute ( (60 x 12) x 9 requests each view = 6480), that way the profile quota shouldn't be exceeded. But this doesn't really make the dashboard realtime anymore.

  • Make a server, which runs the queries(with the increased interval of 1 minute), save the results to a database. The dashboard makes a GET request to the database. This way multiple tv screens should be able to request data.

QUESTION: Could I also make multiple service accounts, and switch to other service account when limit has been reached, or doesn't this fix the profileid limit?

Solution

  • DailylimitExceded can mean one of two things.

    You can only make 10000 request against a single view a day. This quota you are sharing with other developers. So if i install your app and Someone elses app in total there can only be made 10000 requests a day against my Google analytics view and then both apps will get that error. If you are making these requests you should be storing the data in the database so that you don't need to request the same information again. Even though its a different user who is trying to view data on the same view. You are probably not going to be able to track this quota hit in the Google Developer console.

    The second issue is that by default an application can make a max of 50000 requests a day across all views. That means that if you have 5 users and you are making 10000 requests a day for each of them you have reached the limit of your requests. I dont think this is what you are hitting.

    The first quota the user based one there is nothing you can do about that you cant extend it. You need to limit your requests so that you dont block a users account. The second one you can apply for an extension in the Google Developer console it can take a while to get the extension you should apply for it when you have reached around 80% of your current daily quota.

    The main thing here is that you should not be requesting the same data twice. If you have made a request you should be saving it and displaying stored data to your users rather than just requesting it again. That and the real time api you should not be trying to request from that more then every 5 minutes as you will be eating our quota.

    I have suggested to Google several times that the realtime api should be on its own quota and not the same as the reporting api. I am still waiting for them to add this feature.