Search code examples
google-apps-scriptgoogle-sheetshttp-status-code-429

Scrape values from https://app.signalx.ai/gstin-verification/

I am trying to get two values (Legal Name of Business and Principal Place of Business) from a web site https://app.signalx.ai/gstin-verification/ into my google sheet.

If I browse https://app.signalx.ai/gstin-verification/33ABBCS1600H1ZY I am able to see the required values without any login.

But, if I try to fetch the website by the below function, I am not able to get the values. The error returned is 429, but, it occurs even the first time I try.

function get_gst_addrss(){
  var url = ("https://app.signalx.ai/gstin-verification/33ABBCS1600H1ZY");
  var websiteContent = UrlFetchApp.fetch(url).getContentText();
  Logger.log(websiteContent)
}

10:54:46 AM Notice  Execution started
10:54:47 AM Error   
Exception: Request failed for https://app.signalx.ai returned code 429. Truncated server response: <html>
<head><title>429 Too Many Requests</title></head>
<body>
<center><h1>429 Too Many Requests</h1></center>
<hr><center>nginx/1.20.1</cente... (use muteHttpExceptions option to examine full response)
get_gst_addrss  @ macros.gs:
Solution

  • UrlFetchApp.fetch can only pull static websites. This is an expected behaviour of API calls. The website you are pulling is JavaScript generated, which means it is dynamic. To double check this, see below GIF.

    enter image description here

    This is a similar behavior to when pulling data in spreadsheet using IMPORTXML which also has this limitation. Sample posts below: