Search code examples
c#google-analytics-api

Unable to retrieve more than 10k records from Google Analytics


I've developed a windows console application which extracts Google Analytics data and writes to .CSV file. When queried for data of particular date on Google Analytics Query Explorer, it displayed:- "Your query matched 96782 results...".

The problem is when when i query for the data of same date using application, by default it returns only 1000 records and when I set

DataResource.GaResource.GetRequest objRequest.MaxResult more than 10k, then it gives me max to max 10k records.

Is there any way to get records more than 10k.


Solution

  • So what i did is, I set Request.MaxResult = 10000 and Response.ItemsPerPage = 10000. The vital role is played by Request.StartIndex which has pagination mechanism e.g. in 1st loop we'll get 1-10000 rows then 10001-20000....20001-30000 and so on.

    Under loop, request to data of a particular day initially is set to Request.StartIndex = 1 and then consecutively increases to MaxResult + 1 as soon as row count reaches 10000th row e.g.:- Request.StartIndex = 1 then 10001 then 20001....30001...40001 and so on untill Response.Rows == null.

    Here's my code:-

                StringBuilder sbrCSVData = new StringBuilder();
                int intStartIndex = 1;
                int intIndexCnt = 0;
                int intMaxRecords = 10000;
    
                AnalyticsService objAnaSrv = new AnalyticsService(objInit);
                string metrics = "ga:visitors,ga:visits,ga:visitBounceRate,ga:pageviews,ga:pageviewsPerVisit,ga:uniquePageviews,ga:avgTimeOnPage,ga:exits,ga:avgPageLoadTime,ga:goal1ConversionRate";
                DataResource.GaResource.GetRequest objRequest = objAnaSrv.Data.Ga.Get(strProfId, startDate,endDate, metrics);
                objRequest.Dimensions = "ga:visitCount,ga:browser,ga:browserVersion,ga:IsMobile,ga:screenResolution,ga:date,ga:hour";
                objRequest.MaxResults = 10000;
    
                while (true)
                {
                    objRequest.StartIndex = intStartIndex;
                    GaData objResponse = objRequest.Fetch();
                    objResponse.ItemsPerPage = intMaxRecords;
    
                      if (objResponse.Rows != null)
                    {
                        intIndexCnt++;
                        for (int intRows = 0; intRows < objResponse.Rows.Count; intRows++)
                        {
                            IList<string> lstRow = objResponse.Rows[intRows];
                            for (int intCols = 0; intCols <= lstRow.Count - 1; intCols++)
                            {
                                strRowData += lstRow[intCols].ToString() + "|";
                            }
                            strRowData = strRowData.Remove(strRowData.Length - 1, 1);
                            sbrCSVData.Append(strRowData + "\r\n");
                            strRowData = "";
                        }
                        intStartIndex = intIndexCnt * intMaxRecords + 1;
                    }
                    else break;
                }