Search code examples
javascriptnode.jsweb-scrapingbrowserify

Building a webpage that uses Node.js to scrape RSS in the browser

I have some coding experience but am pretty much brand new to Javascript and web development.

I'm trying to build a webpage whose primary functionality involves scraping an RSS feed from the National Weather Service (e.g. https://forecast.weather.gov/MapClick.php?lat=34.0536&lon=-118.2455&FcstType=digitalDWML) and then parsing the output to display various parameters (temperature, humidity, etc.) as text.

I've looked into it and it seems like Node.js has a few different ways to scrape external webpages and return the source as an object. However, from what I understand, Node.js is designed for server-side applications, whereas I want to use it on an actual website that people can access. I'm stuck on the following:

  1. How do I integrate Node.js functionality into the code of a webpage? Do Node.js scripts and other dependencies packaged through something like Browserify automatically "work" in a browser, or is there more to it?
  2. Is there a self-contained Javascript editor that would allow me to utilize Node.js and other dependencies and debug scripts?
Solution

  • Node.js is a means to execute JS programs on the command line. <script> elements are a means to execute JS programs in a webpage in a web browser.

    Browiserify is a tool which takes a set of JS modules that use the CommonJS module system and turns them into a single file which doesn't depend on support for CommonJS. (Node.js uses the CommonJS module system, browsers do not).

    It is possible to write JS code which runs both under Node.js and, using browserify, in a browser. Not all code which runs under Node.js can be converted to run in a browser. The code might depend on APIs which are built into Node.js but not browsers (such as file system access, or network socket support).

    If you have code which depends on those APIs you'll need to either rewrite it so it uses features available in browsers instead, or run it server side and then provide a web service URL for the browser to access with fetch or XMLHttpRequest.