Search code examples
javaspringperlibm-mqprocessing-efficiency

Most efficient way to connect to multiple queues and route their messages to another queue


I'm working with IBM's Websphere MQ queues and I want to connect to multiple of these, browse each ones messages, and route them all to another queue in which a different application will read from.

The application that is reading from the "main" queue is written in Java and using Spring Boot and JMS to process the messages from the queue. I found difficulty in connecting to multiple queues with this application, so the idea was to create a router application, that would connect to multiple queues, clear the "main" queue, and fill it with messages from the collected queues.

I began the "router" application in Java as well and decided to use Spring Boot, but not JMS, and still found some difficulties conceptually with this.

So I'm wondering if Java is right for the job (or maybe I'm missing something fundamental). I'm also pondering a Perl script, but I'm not very (if at all) experienced with the language, but I don't want to discredit it as a viable option.

So which of these two languages could prove to be:

1) Maintainable

2) Somewhat easy to read

3) Efficient

in completing a task such as the one I have described?

Pros and cons of both would be nice.


Solution

  • Apache Camel seemed very promising, but wasn't meshing well with how I had my Spring Boot app set up. I ended up using IBM's MQ Library.

    I basically use these options after setting up the queue

        putMsgOpts = new MQPutMessageOptions();
    
        getFirstMsgOpts = new MQGetMessageOptions();
        getFirstMsgOpts.options = MQConstants.MQGMO_BROWSE_FIRST;
    
        getNextMsgOpts = new MQGetMessageOptions();
        getNextMsgOpts.options = MQConstants.MQGMO_BROWSE_NEXT;
    

    then use my method

    public void transferQueue(MQPropsManager q1) {
        String message = "";
        message = readFromQueue1(q1, getFirstMsgOpts);
        message = verifyMessage(message);   // just a check for empty or null
        writeToQueue2(message);
        q1.decrementMessagesLeftToProcess();  // decrement initial queue depth 
    
        while (q1.getMessagesLeftToProcess() > 0) {
            message = readFromQueue1(q1, getNextMsgOpts);
            message = verifyMessage(message);
            writeToQueue2(message);
            q1.decrementMessagesLeftToProcess();
        }
        closeQueue(q1);
    }
    

    then the two methods that it calls:

    public String readFromQueue1(MQPropsManager q1,
                MQGetMessageOptions getMsgOpts) {
            MQMessage msg = new MQMessage();
            String message = "";
            try {
                q1.getQueue().get(msg, getMsgOpts);
                message = msg.readStringOfCharLength(msg.getMessageLength());
            } catch (IOException ioe) {
            //  Failed to read string retreived from queue: q1.getQueueName()
            } catch (MQException mqe) {
            //  Failed to retreive message from queue: q1.getQueueName()
            }
            return message;
        }
    
    public void writeToQueue2(String message) {
            MQMessage mqMessage = new MQMessage();
            mqMessage.format = MQConstants.MQFMT_STRING;
            mqMessage.messageType = MQConstants.MQMT_DATAGRAM;
    
            try {
                mqMessage.writeString(message);
                q2.getQueue().put(mqMessage, putMsgOpts);
            } catch (IOException ioe) {
            //  Failed to write message: message 
            } catch (MQException mqe) {
            //  Failed to put message: message on to the queue2
            }
        }