How can RabbitMQ Shovel be configured to overwrite the timestamp property with the current time?

Question

For instance:

{myshovel, [
    {sources, ...}
    , {destinations, ...}
    , {queue, <<>>}
    , {ack_mode, on_confirm}
    , {publish_properties, [
        {delivery_mode, 2}
        , {timestamp, now} % this is the line I need to understand how to write
    ]}
    , {publish_fields, [{exchange, <<"">>}, {routing_key, <<"">>}]}
    , {reconnect_delay, 5}
]}

I'm curious how to write the publish_properties in a way so that RabbitMQ Shovel overwrites the timestamp with the current time (as in when the shovel receives the message and shovels it onto the destination queue).

Answer 1

Unfortunately it is not possible to configure the shovel in this way at the time of writing. The shovel configuration, including the publish_properties for forwarded messages, is read when the shovel workers start, and can only contain static content. So whatever value you put into {publish_properties, [{timestamp, TimeStamp}]} will be passed directly to the erlang-client, which is turn will attempt to serialise these (using the amqp_ framing layer).

We are currently planning some improvements to the shovel plugin (such as cluster-wide fail-over and dynamic reconfiguration) and you're not the first person to ask for this feature, so we will consider whether it makes sense to support something specific here (such as setting new timestamps for each processed message) or a general purpose approach to configuring shovel worker's runtime behaviour.