Parsing SNS event Containing S3 events in Lambda

Question

I required multiple lambdas to subscribe to s3 bucket events, this is not supported so the idea is to introduce the SNS instead.

The flow will be S3Event -> SNS -> (X Number of LAMBDA consumers)

The setup and architecture is find, and the Lambda is being triggered correctly with the Event type

SNSEvent with the `ILambdaContext. This is fine.

My problem is when trying to parse the S3 events from the SNSEvent Message.

foreach (var record in snsEvent.Records)
{
    var snsRecord = record.Sns;
    if (string.IsNullOrEmpty(snsRecord.Message))
    {
      logger.Error("Message is empty", record);
      throw new Exception("Message is empty");
    }

  // Object reference not set to an instance of an object.
  var s3Event = JsonSerializer.Deserialize<S3Event>(snsRecord.Message);
  logger.Info($"{s3Event.Records.FirstOrDefault()?.S3.Object.Key}", snsEvent);

  await ProcessParsedEvent(s3Event, context);
} 

The exception is not throw for an empty snsRecord.Message however the cloudwatch is a generic object reference error. However I can't seem to see it.

I suspected there were more than 1 event of a different type which maybe be causing issues with the deserializer, so i added :

var records = snsEvent.Records.Where(x => x.EventSource.ToLowerInvariant() == "aws:s3");

However the same results.. Is there something specific I need to do for deserializing the s3 events ?

Answer 1

The problem is related to the Deserialiazer's property case sensitivity.

The solution in my case was to add the PropertyNameCaseInsensitive in the JsonSerializerOptions set to true.

From here the SNS message (which contains the s3 event message in JSON format, lower case, is able to deserialize correctly.

  var deserializeSettings = new JsonSerializerOptions
  {
      PropertyNameCaseInsensitive = true
  };

  var s3 = JsonSerializer.Deserialize<S3Event>(s3Json, deserializeSettings);