Search code examples
c#html-agility-packselectnodes

C# - SelectNode Return Null in HtmlAgilityPack


I'm trying to get some information from an Instagram profile. I generated XPath with Google Chrome

//*[@id="react-root"]/section/main/article/div/div[1]/div

But , nothing is found . I can find only up

  //*[@id="react-root"]

which is a **<span>**

Nothing more is located , is there any reason? Am I doing something wrong?

var baseURL = "https://www.instagram.com/";

var client = new HtmlWeb();


var paginaPerfil = client.Load(baseURL + "belalao");


var nos = paginaPerfil.DocumentNode.SelectNodes("//*[@id='react-root']/section/main/article/div/div[1]/div");
            var quantidade = nos == null ? 0 : nos.Count;

Solution

  • At this moment in time, https://www.instagram.com/belalao has an empty <span id="react-root"></span>, so your code is working as expected. Perhaps you're expecting the JavaScript which will populate the DOM with the React components will have executed, but HtmlAgilityPack will only analyze the initial HTML you downloaded, not execute JavaScript and analyze afterward.

    View source on the page (not the same as using the DOM inspector), and you'll see what it's seeing.