Search code examples
ssisrabbitmqkingswaysoft

SSIS - RabbitMQ Source missing messages (KingswaySoft)

I am using an SSIS-Package with Kingsway's RabbitMQ components and I am experiencing some strange behavior when loading messages from a queue. I am processing an average of 2,500 messages per day and in some rare cases, single messages seem to be "missing" (my client tells me that they have inserted a specific message into the queue but it does not show up on my end).

I do line counts of everything I receive from the queue and everything that is eventaully inserted into the database - these counts match, so I am fairly certain that I do not lose anything during processing.

I also reroute any errors during processing to error tables, so that no messages will be lost due to faulty content. The only component which does not provide the possibility to reroute errors is the RabbitMQ Source (the RabbitMQ Destination does), so I cannot check if messages are discarded for some reason in rare cases.

This behavior seems to occur randomly for about 0.005% of all messages.

Has anyone experienced a similar issue or can point me in any direction to look for possible errors?

The relevant settings of the components are:

RMQ Connection Manager:

RMQ Connection Manager

RMQ Source:

enter image description here

EDIT:

When a load succeeds, all messages are received. When messages are missing, no messages were transmitted for the batch. For example: There are no cases in which 2 out of 3 messages were transferred. When the error happens, 0 out of 3 messages were transferred. The messages in the queue still get acknowledged and are not transferred in the next batch.

UPDATE: Kingsway has provided an updated build, which I will try out.

Solution

  • I have since received a new build from the Kingsway support team. After testing it for a week, the issue seems to have been resolved.