Search code examples
apache-kafkajson-deserializationmasstransitnodatime

MassTransit with Kafka and NodaTime


I have the following message in C# code:

public interface ResourcePerformance
{
    public string ResourceId { get; }
    public List<TimeSection> TimeSections { get; }
}
public class TimeSection
{
    public Instant PeriodStart { get; set; }
    public Instant PeriodEnd { get; set; }
    public PerformanceStatus PerformanceStatus { get; set; }
    public Duration ProcessingTime { get; set; }
    public double Quantity { get; set; }
}

I want to deserialize such messages from Kafka topic. However when deserializing types from NodaTime library there are errors, for example:

Confluent.Kafka.ConsumeException: Local: Value deserialization error
 ---> Newtonsoft.Json.JsonSerializationException: Error converting value "2:00:00" to type 'NodaTime.Duration'. Path 'timeSections[0].processingTime', line 1, position 330.

I guess it must be something with NodaTime serialization because when I change NodaTime types into object there are no errors reported. I've configured NodaTime for json serializer in RabbitMQ part of the configuration but I don't know how to do it in Kafka part. Currently I have the following configuration:

services.AddMassTransit(x =>
        {
            x.SetKebabCaseEndpointNameFormatter();

            x.UsingRabbitMq((context, cfg) =>
            {

                cfg.ConfigureJsonSerializer( j => j.ConfigureForNodaTime(DateTimeZoneProviders.Tzdb) );
                cfg.ConfigureEndpoints(context);
            });
            
            x.AddRider(rider =>
            {                                     
                rider.AddConsumer<ResourcePerformanceConsumer>();

                rider.UsingKafka((context, k) =>
                {
                                            
                    k.TopicEndpoint<string, ResourcePerformance>("performances-resource", kafkaConsumerGroup , e =>
                    {
                        e.AutoOffsetReset = AutoOffsetReset.Earliest;
                        e.CreateIfMissing(t =>
                        {
                            t.NumPartitions = 4; //number of partitions
                            t.ReplicationFactor = 1; //number of replicas
                        });
                        e.ConfigureConsumer<ResourcePerformanceConsumer>(context);
                    });
                });
            });
        });

        services.AddMassTransitHostedService();

What can I do to deserialize it properly?


Solution

  • I've found the answer on MassTransit discord. In addition to configuring serializer it is necessary to configure deserializer as well:

    cfg.ConfigureJsonDeserializer(j => j.ConfigureForNodaTime(DateTimeZoneProviders.Tzdb) );