Search code examples
web-scrapingnode-red

How to scrape websites in Node-red


I need to scrape a website in Node-red, but I'm not sure about how I should do it.

I know how to scrape websites in Google Spreadsheets, but I don't know how to do the same with the html node in node-red.

This it what I use in Google Spreadsheets:

=index(importxml("https://www.website.dom";"//h2[@class='item-title']");1)


Solution

  • You can use the HTTP Request node to get the content of a page, followed by the HTML node to extract the parts you are interested in.

    The cookbook includes an example recipe for this here: https://cookbook.nodered.org/http/simple-get-request