Remove MessageData from repository after successful consumption

Question

I'm passing a lot of data through RabbitMQ and to offload the queues I'm using the Claim Check pattern to store byte blob outside of the queue itself. It works great, but I have a question about cleanup in the FileSystemMessageDataRepository. Right now I'm specifying TTL for 7 days - in case of any fatal issues with consumer we wish to have this data ready.

// My RabbitMq setup
var messageDataRepository = new FileSystemMessageDataRepository(new DirectoryInfo(massTransitSettings.RabbitMqDataMessageRepositoryPath));
        serviceCollection.AddSingleton<IMessageDataRepository>(messageDataRepository);
        serviceCollection.AddMassTransit(x =>
        {
            x.UsingRabbitMq((context,cfg) =>
            {
                cfg.Host(massTransitSettings.RabbitMqHost, massTransitSettings.RabbitMqVirtualHost, h => {
                    h.Username(massTransitSettings.RabbitMqUsername);
                    h.Password(massTransitSettings.RabbitMqPassword);
                });

                cfg.UseMessageData(messageDataRepository);

                cfg.ConfigureEndpoints(context);
            });
        });

// Message passing and setting up the TTL
var messageData = await messageDataRepository.PutBytes(bufferToSend, TimeSpan.FromDays(7), cancellationToken);
var payload = new MyPayload(messageData, bytesRead);
await bus.Publish(payload, cancellationToken);

The issue i have now is that i don't want MessageData of consumed messages to stick around. After successful consumption i would love to just remove that, but so far i cannot find an option for that. IMessageDataRepository does not have delete methods and I cannot find any option for this to happen automatically. Does anyone have a solution for that?

Answer 1

MassTransit doesn't do any automatic cleanup for you. With the TTL option, the folders are named by the date/hour, so you should be able to write a simple enough script to nuke everything older than a specific time period.