Search code examples
c#cssgoogle-chromehtml-agility-packpuppeteer-sharp

Cannot expand all the div in a page


I'm trying to expand all the divs available inside this page. In particular the div that shows mostra partite, eg:

enter image description here

Looking to the html structure, we have the class: event__header--no-my-games for the collapsed divs, so I did:

// page is the browser page
await page.WaitForSelectorAsync(".event"); // wait ajax completion
var divs = await page.QuerySelectorAllAsync(".event__header--no-my-games > .event__info");

foreach(var cd in divs)
{
   await cd.ClickAsync();
}

the first problem's that the site show 7 divs collapsed, but I get 20 divs on divs variable. Also, when I do:

string html = await page.GetContentAsync();
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(html);

there is no div that has been expanded, what I did wrong?


Solution

  • The following code was working fine:

    await page.WaitForSelectorAsync(".event"); // wait ajax completion
    var divs = await page.QuerySelectorAllAsync(".event__header--no-my-games > .event__info");
    
    foreach(var cd in divs)
    {
       await cd.ClickAsync();
    }
    

    the problem was that the browser opened by Puppeteer had mobile dimension, in that case the selectors that I was looking for was differents.

    I fixed that declaring a Viewport size in the following way:

    Browser = await Puppeteer.LaunchAsync(new LaunchOptions
    {
          Headless = true,
          ExecutablePath = "your chrome path",
          DefaultViewport = new ViewPortOptions { Height = 1280, Width = 800}
    });
    

    this will open the site in desktop mode with correct selectors.

    Hope this help someone.