Azure Service Bus JMS Spring Long running Job

Question

I know this question is asked multiple times, but I am not able to find any good answers.

So I have a AzureServiceBus, which gets new messages from AzureBlob, whenever a new xml file is uploaded to Blob.

In my Spring microService, I have a JMSListner, which listens to this ServiceBus, and when the file upload event is triggered, process the message. For small files, there is no problem and it works perfectly fine. But for the big files(30GB+) the microservice process takes a bit longer time ( more then 10 minutes) and message listener times outs after 5 minutes, the message is placed back in the Queue and soon after the listener again picks up the same message, while the previous process is still running.

due to business limitation, I can process the files in sequence only, so cannot process the next file, until the first is processed.

So I want to know, is

1. it possible to increase the time-out, so that listener should finish the first job to finish.
2. Or If I put the long running process in Async method, then listener should not accept new messages.
3. Or even If the listener accepts the new message, I should be able to acknowledge or put the message back in the queue Gracefully.

Here is my code snippet.

This is my current implementation

  @JmsListener(
    destination = "file-name-queue",
    containerFactory = "jmsListenerContainerFactory"
  )
  public void processMessages(JmsMessage message) throws JMSException{
    
      String fileName = message.getBody(String.class);
      log.info("Process The file {}",fileName);
      // This is the long running Process
      processFullFile(fileName);

      log.info("Process finished for The file {}",fileName);

  }

I am planning to change it to this, if there is no way to increase the time-out.

  @JmsListener(
    destination = "file-name-queue",
    containerFactory = "jmsListenerContainerFactory"
  )
  public void processMessages(JmsMessage message) throws JMSException{

      String fileName = message.getBody(String.class);
      log.info("Received a new file {}",fileName);
      // Check if there 
      if(fileInProgress()) {
        log.info("Existing file already in progress");
        
        //TODO: do something here, so that message gets back in Queue
        // I tried throwing exception here after waiting for sometime, which put the message
        // back in the queue but in the last, and also it was not that graceful
        
      } else {
        log.info("Process The file {}",fileName);
        // This is the long running Process
        aysncProcessFullFile(fileName);
      }
  }

Current Spring properties

spring: jms: servicebus: enabled: true connection-string: Endpoint=sb://XXXXXX.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SHARED_ACCESS_KEY idle-timeout: 2000000 pricing-tier: Standard

In the normal mode I get the following exception for long running process. 10:37:56.246 [org.springframework.jms.JmsListenerEndpointContainer#0-1] INFO c.h.s.c.config.MessageQueueConfig - Under process 10:37:56.286 [org.springframework.jms.JmsListenerEndpointContainer#0-1] INFO o.a.q.jms.JmsLocalTransactionContext - Commit failed for transaction: TX:ID:2897b0c2-8f4e-41d3-9122-f1f89c9f8b83:1:115 10:37:56.287 [org.springframework.jms.JmsListenerEndpointContainer#0-1] INFO o.s.j.c.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection jakarta.jms.TransactionRolledBackException: Transaction is not declared Reference:1cbe9d94-65c4-4a0b-a58d-29bbdf2d616d, TrackingId:6b39f9d4-c914-4ec8-b764-53cbc484b136_G1, SystemTracker:gtm, Timestamp:2023-08-28T08:37:56 [condition = amqp:transaction:unknown-id] at org.apach

Answer 1

You should be able to renew the lock lease when using the ServiceBusClientBuilder.

ServiceBusReceiverAsyncClient receiver = new ServiceBusClientBuilder()
            .connectionString(connectionString)
            .receiver()
            .queueName(queueName)
            .maxAutoLockRenewDuration(Duration.ofMinutes(15))
            .buildAsyncClient();