Scraping an IFrame which has as source a jsp

Question

I'm new to webscraping and I have to do the following: 1. Go to a webpage 2. Find an element 3. Get its value

Now I don't have a problem going to the webpage, that works fine. The problem is that the element I need, actually comes from a jsp which is displayed in an iframe.

I'm trying to get the elemant using HtmlAgilityPack and ScrapySharp.

This is my code :

ScrapingBrowser Browser = new ScrapingBrowser();
Browser.AllowAutoRedirect = true; 
Browser.AllowMetaRedirect = true;

WebPage PageResult = Browser.NavigateToPage(new Uri("[The website I need to go to]"));

HtmlNode myNode = PageResult.Html.CssSelect("#WREPBody").First();
string innerHtml = myNode.InnerHtml;

This works fine. But it returns the following :

<ilayer id="WREPBodyLayer" src="/app014/wrep/rep/gp/jsp/nl/REPGPdata.jsp"></ilayer>

Now the actual element I need is displayed using the REPGPdata.jsp I think. The jsp displays the data as below :

The element I need, would be next to the label "Ondernemingsnr". Does anyone know how I could get access to the element I need?

Answer 1

I found the solution. I just had to browse to the jsp and from that result start scraping again.