Search code examples
.netvb.netawesomiumheadless-browser

Awesomium.Net wrapper class for multi-threaded access only works once


to test this class please install Awesomium.Net sdk from here and add a reference to Awesomium.Core.dll to the visual studio project.

the problem i'm having with this class is, it only works with the first ever instance. subsequent instances do not result in the appropriate events being fired on the WebView object, and no result is returned on later instances. i've been struggling with this for days now and can't seem to figure out why it only works once. please help...

Imports Awesomium.Core

Public Class Browser

    Private Shared Thread As Threading.Thread
    Private Shared CoreIsRunning As Boolean = False
    Dim RenderedHTML As String = ""
    Dim RenderingDone As Boolean = False

    Private Shared Sub AwesomiumThread()
        WebCore.Initialize(New WebConfig With {.LogLevel = LogLevel.None}, False)
        WebCore.Run(Sub(s, e)
                        CoreIsRunning = True
                    End Sub)
    End Sub

    Shared Sub DeInit()
        WebCore.Shutdown()
        Thread = Nothing
    End Sub

    Shared Sub Init()
        If Not WebCore.IsInitialized And IsNothing(Thread) Then
            Thread = New Threading.Thread(AddressOf AwesomiumThread)
            Thread.Start()
        End If
    End Sub

    Sub New()
        Init()
    End Sub

    Function GetRenderedHTML(URL As String) As String
        Do Until CoreIsRunning
            Task.Delay(300).Wait()
        Loop

        WebCore.QueueWork(Sub() RenderHTML(URL))

        Dim startTime As Date = Date.UtcNow
        Do Until RenderingDone = True
            If Date.UtcNow.Subtract(startTime).TotalSeconds >= 30 Then
                'view.Dispose()
                'view = Nothing
                'session.Dispose()
                'session = Nothing
                Exit Do
            End If
            Task.Delay(1000).Wait()
        Loop

        If String.IsNullOrEmpty(RenderedHTML) Then
            Throw New Exception("Rendering failed!")
        End If

        Return Me.RenderedHTML
    End Function

    Private Sub RenderHTML(URL As String)
        Dim session As WebSession = WebCore.CreateWebSession(New WebPreferences With {.LoadImagesAutomatically = False, .LocalStorage = False, .Plugins = False, .RemoteFonts = False, .WebAudio = False, .CanScriptsOpenWindows = False, .DefaultEncoding = "utf-8"})
        Dim view As WebView = WebCore.CreateWebView(1100, 600, session, WebViewType.Offscreen)
        RenderingDone = False
        AddHandler view.LoadingFrameComplete, Sub(s, e)
                                                  If e.IsMainFrame Then
                                                      Me.RenderedHTML = view.ExecuteJavascriptWithResult("document.documentElement.outerHTML").ToString
                                                      RenderingDone = True
                                                      RemoveHandler view.LoadingFrameFailed, Nothing
                                                      view.Dispose()
                                                      view = Nothing
                                                      session.Dispose()
                                                      session = Nothing
                                                  End If
                                              End Sub
        AddHandler view.LoadingFrameFailed, Sub(s, e)
                                                If e.IsMainFrame Then
                                                    Me.RenderedHTML = ""
                                                    RenderingDone = True
                                                    RemoveHandler view.LoadingFrameFailed, Nothing
                                                    view.Dispose()
                                                    view = Nothing
                                                    session.Dispose()
                                                    session = Nothing
                                                End If
                                            End Sub
        view.Source = URL.ToUri
    End Sub

End Class

I'm using the above Browser class in a multi-threaded winforms app which is a web scraper like so:

Dim A as New Browser
Dim ResultA As String = A.GetRenderedHTML("http://google.com")

Dim B as New Browser
Dim ResultB As String = B.GetRenderedHTML("http://google.com")

A.GetRenderedHTML works but B.GetRenderedHTML fails when called straight after. I.E. only the first ever instance of Browser class works.


Solution

  • Thanks everyone who commented.

    I believe I have finally come up with a working thread safe wrapper class for Awesomium.Net

    I've posted the final working source code over at github. You can check it out at the link below:

    https://github.com/dj-nitehawk/ThreadSafeAwesomium.Net/blob/master/Browser.vb

    Leaving this answer here for interested parties :-)

    Cheers!