Google Fusion Tables Quota Exceeded when importing data

Question

I fill google fusion tables . I've about 2mio entries which should be added to the fusion tables.

I use Google.Apis.Fusiontables.v2 1.10.0.50

This worked for some monthes. but now I get a quota exceeded exception.

Google.Apis.Requests.RequestError Quota Exceeded [403] Errors [ Message[Quota Exceeded] Location[ - ] Reason[quotaExceeded] Domain[usageLimits] ]

As there are more data this isn't that surprising.

But when I look at the quota on https://console.developers.google.com/apis/api/fusiontables/quotas I can't see any limitations were hit.

Also in the global quota dashboard I can't see any limits I hit.

I thought that I may hit the 250MB per table. but when I download the datasource the csv only has 40 MB But I think this are only read quotas not write?

        private static void WriteItem()
        {
            await WriteAsync("map_data", index.AllEdges<BusEdge<TId>>().Where(x => x.MinFrequentPriceInEur > 0), (busEdge, csvWriter) =>
            {
                WriteItem(busEdge, csvWriter);

                itemHandled();
            });
        }

    private static void WriteItem(BusEdge<TId> busEdge, CsvWriter csvWriter)
    {
        csvWriter.WriteField(Source.Id.ToString());
        csvWriter.WriteField(sink.Id.ToString());

        csvWriter.WriteField(string.Format(CultureInfo.InvariantCulture, "{0} {1}",
            sink.Geolocation.Latitude,
            sink.Geolocation.Longitude));

        csvWriter.WriteField(sink.Population);

        foreach (Language language in Languages.AllLanguages)
        {
            csvWriter.WriteField(sink.DisplayName[language], true);
        }

        var priceWithDate =
            busEdge.Prices.GroupBy(x => x.Value.MinimumFrequentInEur)
                .OrderByDescending(x => x.Count()).SelectMany(x => x)
                .OrderBy(x => x.Key)
                .FirstOrDefault();

        double price = priceWithDate.Value.MinimumFrequentInEur;

        csvWriter.WriteField(Math.Round(price, 2));
        csvWriter.WriteField(Math.Round(CurrencyInfo.EUR.ConvertMoney(price, CurrencyInfo.GBP), 2));
        csvWriter.WriteField(Math.Round(CurrencyInfo.EUR.ConvertMoney(price, CurrencyInfo.HRK), 2));
        csvWriter.WriteField(Math.Round(CurrencyInfo.EUR.ConvertMoney(price, CurrencyInfo.PLN), 2));
        csvWriter.WriteField(priceWithDate.Key.ToString("s"));

  public static async Task WriteAsync<T>(string tableName, IEnumerable<T> list, Action<T, CsvWriter> writerAction)
        {
            using (FusiontablesService service = Connect())
            {
                var tables = service.Table.List().Execute();
                var tableId = tables.Items.Where(x => x.Name == tableName).Select(x => x.TableId).First();

                MemoryStream memoryStream = new MemoryStream();

                bool hasWritten = false;

                Func<Task> uploadAsync = async () =>
                {
                    if (memoryStream.Length > 0)
                    {
                        memoryStream.Position = 0;

                        ResumableUpload<string> request =
                            hasWritten ?
                                (ResumableUpload<string>)service.Table.ImportRows(tableId, memoryStream, "application/octet-stream") :
                                (ResumableUpload<string>)service.Table.ReplaceRows(tableId, memoryStream, "application/octet-stream");

                        var response = await request.UploadAsync();

                        if (response.Exception != null)
                        {
                            throw response.Exception;
                        }

                        memoryStream = new MemoryStream();

                        hasWritten = true;
                    }
                };

                using (StreamWriter writer = new StreamWriter(memoryStream, Encoding.UTF8, 1024, true))
                {
                    using (CsvWriter csvWriter = new CsvWriter(writer, new CsvConfiguration { Delimiter = "," }))
                    {
                        foreach (T item in list)
                        {
                            writerAction(item, csvWriter);

                            csvWriter.NextRecord();

                            writer.Flush();

                            if (memoryStream.Length >= Limit)
                            {
                                await uploadAsync();
                            }
                        }
                    }
                }

                await uploadAsync();
            }

Any advice how I can find out the specific limit ? and any recommandations on improvements in the code ?

Answer 1

I emailed the support. They told me they will have look. I didn't hear anything new, but the day after that it worked without any change.

Now I have about 20% more data and it still workds. so it looks like it was a google bug