Search code examples
azureazure-eventhubqpid

EventHubConsumerClient Apache Qpid memory leak?

I am reading events from an Azure EventHub cluster synchronously via the receiveFromPartition method on the EventHubConsumerClient class.

I create the client once like so:

EventHubConsumerClient eventHubConsumerClient = new EventHubClientBuilder()
            .connectionString(eventHubConnectionString)
            .consumerGroup(consumerGroup)
            .buildConsumerClient());

I then just use a ScheduledExecutorService to retrieve events every 1.5s via:

IterableStream<PartitionEvent> receivedEvents = eventHubConsumerClient.receiveFromPartition(
                partitionId, 1, eventPosition);

The equivalent logic in V3 of the SDK worked fine (using PartitionReceivers), but now I am seeing OOMs in my JVM.

Running a profiler against a local version of the logic I see the majority of the heap (90%, mainly in OG) is being taken up by byte[]s, referenced by org.apache.qpid.proton.codex.CompositeReadableBuffer. This pattern is not present when I profile the V3 logic.

What could be causing a leak of the AMQP messages here, do I need to interact with the SDK further, for example close a connection that I'm not aware of after each call?

Any advise would be very appreciated, thanks!

Solution

  • Turns out it was a bug, solved here: https://github.com/Azure/azure-sdk-for-java/issues/13775