Search code examples
rabbitmqrabbitmq-shovel

RabbitMQ Shovel Stuck in 'Terminated' Status

We have an issue where once in a while, a dynamic shovel (created via the HTTP API: /api/parameters/shovel/) with src-delete-after set to queue-length finishes and then instead of being deleted, gets stuck in a terminated status.

Subsequent attempts to delete the shovel by any of the following methods are unsuccessful:

  1. Posting a DELETE to /api/parameters/shovel/
  2. rabbitmqctl delete_shovel
  3. rabbitmqctl clear_parameter -p <vhost> shovel <shovel_name>

The shovel doesn't even appear in the 'Shovel Management' section of the RabbitMQ admin UI.

The only way we could get rid of that stuck shovel is by restarting RabbitMQ.

Is anyone else having this issue? If so, how do we clear the shovel without having to restart the cluster? Also, is it possible to prevent this from happening via configuration?

Thanks!

PS:

  1. RabbitMQ version: 3.4.4
  2. Running a 2 node cluster (will be making it a 3 node cluster shortly due to obvious issue we could face in the case of a network partition).

RabbitMQ Shovel Stuck in 'Terminated' Status

Solution

  • You are using a very, very old version of RabbitMQ. Please upgrade to the latest version (3.7.6) and be sure to use Erlang 20.3.X (not 21).