Search code examples
c#.netweb-scrapinghtml-agility-pack

HTML Agility Pack - Can only load xml document from file system, not from web


I've used HAP successfully before, downloading xhtml pages from web. However, now I'm trying to load and parse xml documents. HAP will only load xml documents that are located on my file system, "C:\xml\MyXml.xml" for instance. It will not load it from web (http://www.web.com/doc.xml). Using Fiddler, I can see that HAP is actually requesting the xml documents over the web, and the server also responds with the xml document. However, it stops there, nothing get parsed. The HtmlDocument is empty, no ChildNodes or anything. When loading from file system, it get parsed successfully to a HtmlDocument.

Any ideas?


Solution

  • If you are using just the XML (and not (X)HTML) then you don't need to use HAP as .Net has comprehensive XML processing built in:

    String PostUrl = "http://www.web.com/doc.xml"; 
    WebResponse webResponse = WebRequest.Create(PostUrl).GetResponse();
    StreamReader sr = new StreamReader(webResponse.GetResponseStream());
    
    String Result = sr.ReadToEnd().Trim();
    
    XmlDocument xdoc = new XmlDocument(); xdoc.LoadXml(Result);