Search code examples
c#webclient

Get generated html from webpage


I am trying to do some web scraping from a webpage. But the there is information on the screen, that i cannot find in the source-code. However if I manually save the webpage as a local file, i get the static generated page. Can I do that in C#?

The below code only gives me the source of the webpage, but not the values I am interested in.

WebClient client = new WebClient();
string sPageHtml = null;
string sUrl = @"http://www.nasdaqomxnordic.com/bonds/denmark/microsite?Instrument=XCSE0%3A5RD27SSA50";

sPageHtml = client.DownloadString(sUrl);

Solution

  • I suggest you use a dedicated tool like the HtmlAgilityPack Nuget package :

    Package description: this is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" malformed HTML. The object model is very similar to what proposes System.Xml, but for HTML documents (or streams).

    I guess it would be easier for you to grab what you're interested in.