Large http payloads aren't getting sent to the Serilog Http Sink endpoint in .NET Core 3.1

Question

Summary

I'm having trouble posting from Serilog (Http Sink) to my custom .NET Core 3.1 WebAPI endpoint when the logging data is large. If I remove some log data when I do the logging, then Serilog sinks properly with my WebAPI endpoint.

My Configuration

            new LoggerConfiguration()
                     .Enrich.FromLogContext()
                     .WriteTo.Http(httpPath, httpClient: new CustomHttpClient(), batchPostingLimit: int.MaxValue, queueLimit: int.MaxValue)
                     .CreateLogger();

My Custom Http Client

public class CustomHttpClient : IHttpClient
{
    private readonly HttpClient c_httpClient;

    public CustomHttpClient()
    {
        c_httpClient = new HttpClient
        {
            MaxResponseContentBufferSize = 2147483647L
        };
    }

    public void Configure(IConfiguration configuration)
    {
        
    }

    public Task<HttpResponseMessage> PostAsync(string requestUri, HttpContent content) => c_httpClient.PostAsync(requestUri, content);

    public void Dispose() => c_httpClient?.Dispose();
}

What actually does the logging

            var exceptionModel = new AppMonModel
            {
                Application = "SerilogMvc Sample Application",
                Message = ex.Message,
                Source = "SerilogMvc.HomeController.Index",
                StackTrace = ex.StackTrace,
                InnerException = ex.InnerException?.StackTrace,
                Details = "Sample details here",
                InsertDate = DateTime.Now,
                Severity = 100,
                UserDescription = "Keyvan User",
                ScreenshotBase64String = Convert.ToBase64String(System.IO.File.ReadAllBytes("C:/SamplePath/Untitled.png"))
            };
            
            c_logger.LogError(ex, "{exceptionModel}", exceptionModel);

My Endpoint

    [HttpPost("log")]
    [DisableRequestSizeLimit]
    public void Log([FromBody] object logEvents) { ... }

Serilog Error

Event JSON representation exceeds the byte size limit of 262144 set for this sink and will be dropped;

Issue

When I remove ScreenshotBase64String = Convert.ToBase64String(System.IO.File.ReadAllBytes("C:/SamplePath/Untitled.png")) from my exceptionModel object, I see the error in my WebAPI endpoint. As soon as I add it back in, it doesn't even hit the endpoint.

Please let me know if you need additional details. I'd be more than glad to provide them.

Answer 1

The answer was quite simple after turning on Self logging. This is the change I needed to make to increase the batch formatter size:

        var defaultBatchFormatter = new DefaultBatchFormatter(batchFormatterSize);

        Log.Logger = new LoggerConfiguration()
                     .MinimumLevel.Error()
                     .Enrich.FromLogContext()
                     .WriteTo.Http(httpPath, batchFormatter: defaultBatchFormatter)
                     .CreateLogger();

The batch formatter size needed to be increased.