Search code examples
c#chromiumcefsharp

Waiting for javascript-rendered page loading in CefSharp


I started to learn CefSharp. It is a convinient tool for web scraping. But I have a problem. I want to wait for a page to be loaded. But the page loading is completed by ajax. And I'm waiting for some html tags. I wrote like this.

public ChromiumWebBrowser brser;
Thread main_thread;
public Form1()
{
    brser = new ChromiumWebBrowser("some_url_here");
    panelBrowser.Controls.Add(brser);
    brser.Dock = DockStyle.Fill;
    main_thread = new Thread(mainStart);
    main_thread.Start();
}
private void mainStart()
{
    brser.LoadingStateChanged += LoadingStateChanged;
    brser.Load("other_url_here");
}
private async void LoadingStateChanged(object sender, LoadingStateChangedEventArgs e)
{
//... Omitted Code ...
    Context.LogInfo("[LOG]\t----");
    frame = brser.GetBrowser().GetFrame("iframe1");
    if(frame == null)
    {
        Context.LogInfo("[LOG]\tiframe1 is null. return");
        return;
    }
    int nretry = 100;
    try
    {
        while (nretry > 0)
        {
            html = await Task.Run(frame.GetSourceAsync);
            if (html.Contains("<ul class=\"some_class_to_be_loaded\">"))
            {
                break;
            }
            nretry--;
            Thread.Sleep(100);
        }
        if (nretry == 0)
        {
            MessageBox.Show("Connection Error");
            try
            {
                main_thread.Abort();
                return;
            }
            catch (Exception ex)
            {
                return;
            }
        }
    }
    catch
    {
        return;
    }
}

But it does not work. I debugged it. But when the thread sleeps, the page loading sleeps too. As a result, waiting 'loading complete event' failed. Please help me.


Solution

  • You can wrap your waiting part by delegate. Like this.

    Invoke(new Action(async delegate ()
    {
    //waiting code
        int nretry = 100;
        while (nretry > 0){
    //...
            Thread.Sleep(100);
        }
    }