SelectNodes in HtmlAgilityPack always return null

Question

I'm trying to get data from this site: https://www.hltv.org/results, and it doesnt work. SelectNodes always returns null, I've tried using xpath, full xpath and so on and different things other than HtmlAgilityPack. Maybe its not the HtmlAgilityPack maybe the problem is in the tags. Please take a look at the code and view also the tags to see if if i copied it right:

    HtmlWeb web = new HtmlWeb();
    var doc = web.Load("https://www.hltv.org/results");
    var teams = doc.DocumentNode.SelectNodes("//*[@class='team team-won']");

Please Help, Thanks!

Answer 1

HtmlAgilityPack does not handle javascript, so you can use selenium to store the html source code via webDriver.

IWebDriver webDriver = new ChromeDriver();

webDriver.Url = "https://www.hltv.org/results";
var pageSource = webDriver.PageSource;

var doc = new HtmlDocument();
doc.LoadHtml(pageSource);

 var xPath = @"//*[@class='team team-won']";

 var node = doc.DocumentNode.SelectSingleNode(xPath);

 Console.WriteLine(node.InnerText);

Make sure that you have already downloaded and stored in same directory the webDriver.(i used chrome driver downloaded from here https://chromedriver.chromium.org/downloads)