Search code examples
cachinggoogle-apps-scriptweb-scrapinggoogle-sheetsurlfetch

Exception: argument too large: value Google Script


I'm trying to scrape a website & put the value in cache so I don't hit the daily limit of UrlFetchApp

Here is the script I did:

/**
 * Scrape URL, return whatever you choose with jquery-style selectors. 
 Dependency: cheeriogs, see https://github.com/fgborges/cheeriogs
 *
 * @param {url} valid start-url
 * @return result (array values)
 *
 * @customfunction
 */
function scrapercache(url) {
    var result = [];
    var description;
    var options = {
        'muteHttpExceptions': true,
        'followRedirects': false,
    };
  
var cache = CacheService.getScriptCache();
var properties = PropertiesService.getScriptProperties();

try {  
  let res = cache.get(url);

  if (!res) {
    // trim url to prevent (rare) errors
    url.toString().trim();
    var r = UrlFetchApp.fetch(url, options);
    var c = r.getResponseCode();

    // check for meta refresh if 200 ok
    if (c == 200) {
      var html = r.getContentText();
      cache.put(url, "cached", 21600);
      properties.setProperty(url, html);

      var $ = Cheerio.load(html); // make sure this lib is added to your project!

      // meta description
      if ($('meta[name=description]').attr("content")) {
        description = $('meta[name=description]').attr("content").trim();
      }
    }
  
    result.push([description]);    
  }
} 
catch (error) {
  result.push(error.toString());
} 
finally {
  return result;
}
 
}

but when I call the function like that:

=scrapercache("https://www.gurufocus.com/term/total_freecashflow/nyse:ABBV/Free-Cash-Flow")

I get the error message:

Exception: argument too large: value 

Anyone can help me please? Thank you :) Gabriel


Solution

  • As written in the official documentation,

    The maximum length of a key is 250 characters. The maximum amount of data that can be stored per key is 100KB.

    If the size of the data put in cache exceeds any of the above limitations, the error

    Exception: argument too large

    is shown. In your case, value exceeds 100KB. Solution would be to cache only necessary data or don't cache at all depending on your specific needs.