Async NullRefrenceException from Html Agility Pack's LoadFromWebAsync

Question

I'm using Html Agility Pack to get meta tags from URLs. I was using HtmlWeb's Load, but I found it to be taking too long and thought that I could probably speed up performance using an async method instead. However, when I tried to use LoadFromWebAsync I get a NullReferenceException.

Using Load

public static MetaTags GetMetaData(string url)
{
    // This works 
    var webGet = new HtmlWeb();
    var document = webGet.Load(url);
    var metaTags = document.DocumentNode.SelectNodes("//meta");
    MetaTags metaInfo = new MetaTags(url);
    //more code
}
        

Using LoadFromWebAsync

public static async Task<MetaTags> GetMetaData(string url)
{
    // This doesn't work 
    var webGet = new HtmlWeb();
    var document = await webGet.LoadFromWebAsync(url); //breaks here
    var metaTags = document.DocumentNode.SelectNodes("//meta");
    MetaTags metaInfo = new MetaTags(url);
    //more code
}

I am using this method to get the url data to pass to a NewsCard component which I attempt to do like:

<p
@news.Title
</p>

@code {
    public MetaTags news { get; set; }
    protected override async Task OnInitializedAsync()
    {
            news = await MetaScraper.GetMetaData(" https://www.denverpost.com/2020/10/03/president-trump-fight-against-covid-19-update/");
    }
}

Why am I getting this error, and what can I do to fix it?

Edit: Im not sure what is null, I'm working in Blazor and when it breaks it exits out to _Host.cshtml on this line which existed since I created the project: . I think it might have something to do with me attempting to load the component before it has anything to fill.

Stack Trace:

ThinBlueLieB.Components.NewsCard.<BuildRenderTree>b__0_0(RenderTreeBuilder __builder2)
Microsoft.AspNetCore.Components.Rendering.RenderTreeBuilder.AddContent(int sequence, RenderFragment fragment)
Syncfusion.Blazor.Cards.SfCard.BuildRenderTree(RenderTreeBuilder __builder)
Microsoft.AspNetCore.Components.ComponentBase.<.ctor>b__6_0(RenderTreeBuilder builder)
Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment)
Microsoft.AspNetCore.Components.RenderTree.Renderer.RenderInExistingBatch(RenderQueueEntry renderQueueEntry)
Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
Microsoft.AspNetCore.Components.Rendering.HtmlRenderer.HandleException(Exception exception)
Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessPendingRender()
Microsoft.AspNetCore.Components.RenderTree.Renderer.AddToRenderQueue(int componentId, RenderFragment renderFragment)
Microsoft.AspNetCore.Components.ComponentBase.StateHasChanged()
Microsoft.AspNetCore.Components.ComponentBase.CallOnParametersSetAsync()
Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
Microsoft.AspNetCore.Components.Rendering.HtmlRenderer.HandleException(Exception exception)
Microsoft.AspNetCore.Components.RenderTree.Renderer.AddToPendingTasks(Task task)
Microsoft.AspNetCore.Components.Rendering.ComponentState.SetDirectParameters(ParameterView parameters)
Microsoft.AspNetCore.Components.RenderTree.Renderer.RenderRootComponentAsync(int componentId, ParameterView initialParameters)
Microsoft.AspNetCore.Components.Rendering.HtmlRenderer.CreateInitialRenderAsync(Type componentType, ParameterView initialParameters)
Microsoft.AspNetCore.Components.Rendering.HtmlRenderer.RenderComponentAsync(Type componentType, ParameterView initialParameters)
Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext+<>c__11<TResult>+<<InvokeAsync>b__11_0>d.MoveNext()
Microsoft.AspNetCore.Mvc.ViewFeatures.StaticComponentRenderer.PrerenderComponentAsync(ParameterView parameters, HttpContext httpContext, Type componentType)
Microsoft.AspNetCore.Mvc.ViewFeatures.ComponentRenderer.PrerenderedServerComponentAsync(HttpContext context, ServerComponentInvocationSequence invocationId, Type type, ParameterView parametersCollection)
Microsoft.AspNetCore.Mvc.ViewFeatures.ComponentRenderer.RenderComponentAsync(ViewContext viewContext, Type componentType, RenderMode renderMode, object parameters)
Microsoft.AspNetCore.Mvc.TagHelpers.ComponentTagHelper.ProcessAsync(TagHelperContext context, TagHelperOutput output)
Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.<RunAsync>g__Awaited|0_0(Task task, TagHelperExecutionContext executionContext, int i, int count)
ThinBlueLieB.Pages.Pages__Host.<ExecuteAsync>b__17_1() in _Host.cshtml
+
        <component type="typeof(App)" render-mode="ServerPrerendered" />        
Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext.SetOutputContentAsync()
ThinBlueLieB.Pages.Pages__Host.ExecuteAsync() in _Host.cshtml
+
    Layout = null;
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, bool invokeViewStarts)
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, string contentType, Nullable<int> statusCode)
Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, string contentType, Nullable<int> statusCode)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|29_0<TFilter, TFilterAsync>(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext<TFilter, TFilterAsync>(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
Microsoft.VisualStudio.Web.BrowserLink.BrowserLinkMiddleware.ExecuteWithFilter(IHttpSocketAdapter injectScriptSocket, string requestId, HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Answer 1

The issue was that I was attempting to show the data before it was loaded. This was because I was using async but not providing something to show while it was awaiting a result. It can be fixed by doing:

if (news.HasData == false) {
    <p>Loading</p>
}
else{
    <p>
    @news.Title
    </p>
}
@code {
    public MetaTags news { get; set; }
    protected override async Task OnInitializedAsync()
    {
        news = new MetaTags(newsUrl); //initialized with HasData = false
        news = await MetaScraper.GetMetaData(newsUrl); //Filling out with data later. 
    }
}