Search code examples
activemq-artemishawtio

ActiveMQ Artemis Web Console - Unsupported message body

I am encountering some issues with large messages in my ActiveMQ Artemis 2.6.0 setup, and I would appreciate your insights.

Message Visibility in the Web Console:

Messages exceeding 100 KiB in size are not displayed in the console, e.g.:

Snapshot showing unsupported message in hawtio console

I have configured the min-large-message-size to 1000 KiB in the cluster connection settings, which should ensure that only messages above this threshold are stored in the large message directory. However, it seems this configuration is not being applied as expected.

Message Encoding in Large Messages Directory:

Another pattern I am observing is that some messages persisted in the large message directory include binary characters, e.g.:

�x<?xml version="1.0" encoding="UTF-8"?> 
<Message

I am unclear whether this is expected behavior or if there are configuration issues that need to be revisited.

REST API Browse Behavior:

When using the REST API via the curl command to browse messages and take a message dump, messages stored as large messages are not being retrieved.

Should the browse operation be able to access and display large messages? If not, is there a different method or configuration required to handle this? See snapshot 2

Are there any alternative tools to see these kind of messages and take a back up as XML's that are decoded and doesn't add these binary characters. Note: These special characters in the message XML are not added by client this was tested

Could someone please shed some light on these issues or recommend any settings or configurations I should revisit to resolve them?

broker.xml:

<?xml version='1.0'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->
<configuration xmlns="urn:activemq"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:xi="http://www.w3.org/2001/XInclude"
               xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">

   <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="urn:activemq:core ">

      <name>artemis-master</name>

      <persistence-enabled>true</persistence-enabled>

      <journal-type>ASYNCIO</journal-type>

      <paging-directory>/data-volume/artemis-master/data/paging</paging-directory>

      <bindings-directory>/data-volume/artemis-master/data/bindings</bindings-directory>

      <journal-directory>/data-volume/artemis-master/data/journal</journal-directory>

      <large-messages-directory>/data-volume/artemis-master/data/large-messages</large-messages-directory>

      <journal-datasync>true</journal-datasync>

      <journal-min-files>10</journal-min-files>

      <journal-pool-files>100</journal-pool-files>

      <journal-file-size>10M</journal-file-size>

      <journal-buffer-size>2M</journal-buffer-size>

      <journal-buffer-timeout>20000</journal-buffer-timeout>

      <journal-max-io>4096</journal-max-io>

     <connectors>
      <connector name="artemis-master">tcp://x.x.x.x:61616</connector>
      <connector name="artemis-slave">tcp://x.x.x.x:61616</connector>
     </connectors>

      <cluster-connections>
            <cluster-connection name="MQ_Cluster">
               <address></address>
               <connector-ref>artemis-master</connector-ref>
               <check-period>1000</check-period>
               <connection-ttl>60000</connection-ttl>
               <min-large-message-size>1000000</min-large-message-size>
               <call-timeout>30000</call-timeout>
               <retry-interval>500</retry-interval>
               <retry-interval-multiplier>1.0</retry-interval-multiplier>
               <max-retry-interval>5000</max-retry-interval>
               <initial-connect-attempts>-1</initial-connect-attempts>
               <reconnect-attempts>-1</reconnect-attempts>
               <use-duplicate-detection>true</use-duplicate-detection>
               <message-load-balancing>ON_DEMAND</message-load-balancing>
               <max-hops>1</max-hops>
               <confirmation-window-size>32000</confirmation-window-size>
               <call-failover-timeout>30000</call-failover-timeout>
               <notification-interval>1000</notification-interval>
               <notification-attempts>2</notification-attempts>
               <static-connectors>
                  <connector-ref>artemis-slave</connector-ref>
               </static-connectors>
            </cluster-connection>
      </cluster-connections>

      <disk-scan-period>5000</disk-scan-period>


      <max-disk-usage>90</max-disk-usage>

      <critical-analyzer>true</critical-analyzer>

      <critical-analyzer-timeout>120000</critical-analyzer-timeout>

      <critical-analyzer-check-period>60000</critical-analyzer-check-period>

      <critical-analyzer-policy>HALT</critical-analyzer-policy>

      <global-max-size>7Gb</global-max-size>

      <acceptors>

         <!-- Acceptor for every supported protocol -->
         <acceptor name="artemis">tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;keepAlive=true</acceptor>

         <!-- AMQP Acceptor.  Listens on default AMQP port for AMQP traffic.-->
         <acceptor name="amqp">tcp://0.0.0.0:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300</acceptor>

         <!-- STOMP Acceptor. -->
         <acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor>

         <!-- HornetQ Compatibility Acceptor.  Enables HornetQ Core and STOMP for legacy HornetQ clients. -->
         <acceptor name="hornetq">tcp://0.0.0.0:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true</acceptor>

         <!-- MQTT Acceptor -->
         <acceptor name="mqtt">tcp://0.0.0.0:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor>

      </acceptors>


      <cluster-user>admin</cluster-user>

      <cluster-password>admin</cluster-password>

      <ha-policy>
         <replication>
            <master>
              <check-for-live-server>true</check-for-live-server>
        </master>
         </replication>
      </ha-policy>

      <security-settings>
         <security-setting match="#">
            <permission type="createNonDurableQueue" roles="amq"/>
            <permission type="deleteNonDurableQueue" roles="amq"/>
            <permission type="createDurableQueue" roles="amq"/>
            <permission type="deleteDurableQueue" roles="amq"/>
            <permission type="createAddress" roles="amq"/>
            <permission type="deleteAddress" roles="amq"/>
            <permission type="consume" roles="amq"/>
            <permission type="browse" roles="amq"/>
            <permission type="send" roles="amq"/>
            <permission type="manage" roles="amq"/>
         </security-setting>
      </security-settings>

      <address-settings>
         <address-setting match="activemq.management#">
            <dead-letter-address>DLQ</dead-letter-address>
            <expiry-address>ExpiryQueue</expiry-address>
            <redelivery-delay>0</redelivery-delay>
            <max-size-bytes>-1</max-size-bytes>
            <message-counter-history-day-limit>10</message-counter-history-day-limit>
            <address-full-policy>PAGE</address-full-policy>
            <auto-create-queues>true</auto-create-queues>
            <auto-create-addresses>true</auto-create-addresses>
            <auto-create-jms-queues>true</auto-create-jms-queues>
            <auto-create-jms-topics>true</auto-create-jms-topics>
         </address-setting>
         <address-setting match="#">
            <dead-letter-address>DLQ</dead-letter-address>
            <expiry-address>ExpiryQueue</expiry-address>
            <redelivery-delay>0</redelivery-delay>
            <max-size-bytes>-1</max-size-bytes>
            <message-counter-history-day-limit>10</message-counter-history-day-limit>
            <address-full-policy>PAGE</address-full-policy>
            <auto-create-queues>false</auto-create-queues>
            <auto-create-addresses>false</auto-create-addresses>
            <auto-create-jms-queues>false</auto-create-jms-queues>
            <auto-create-jms-topics>false</auto-create-jms-topics>
         </address-setting>
      </address-settings>

      <addresses>
         <address name="DLQ">
            <anycast>
               <queue name="DLQ" />
            </anycast>
         </address>
         <address name="ExpiryQueue">
            <anycast>
               <queue name="ExpiryQueue" />
            </anycast>
         </address>

      </addresses>
   </core>
</configuration>

enter image description here

Solution

  • I believe all the behavior you're seeing is expected. Let me explain...

    The main goal of large message support is to avoid ever loading the full message into memory. Any message whose body exceeds the relevant minLargeMessageSize limit will be sent in chunks as a "large" message. These chunks are received by the broker and streamed to disk as a file in the large messages directory. This feature allows both the broker and the Core client to handle arbitrarily large messages with a small memory footprint.

    Transmitting the entire body of a large message via a management method (e.g. for viewing in the web console) would defeat this purpose. Consider a 10GB message. If 10GB of data was sent to and displayed in the web console that wouldn't be a very good user experience. It would take a long time to load, and there's no guarantee it would even be readable by a human. It would also risk exhausting the memory of the broker's JVM since the web console can't receive the data in chunks. This is why the web console displays, "Unsupported message body type..."

    The default size beyond which a message is considered "large" by the ActiveMQ Artemis Core client (including the Core JMS client) is 100KiB (102,400 bytes). Generally speaking, this is configured by the minLargeMessageSize connection URL parameter on the client itself. The min-large-message-size parameter on any cluster-connection defined in broker.xml only applies to messages sent over the cluster connection. It doesn't apply to remote clients sending messages to the broker.

    The files in the large messages directory are for internal use and, aside from the message body data, may contain additional meta-data related to internal broker functions. I wouldn't recommend using these files directly.

    The browse management method on the queue which is available via the REST API is the same method used by the web console to browse the queue. It will not display the body of a large message as noted previously.

    If you want to receive the entire message body without removing the message from the broker then you can create a JMS consumer using the client-acknowledge mode and simply not acknowledge the message. When you close your consumer all the unacknowledged messages will still be on the broker.

    You might also consider changing the producer's configuration (i.e. increasing minLargeMessageSize) so that you avoid having any large messages in the first place. Of course, this will use additional memory on the client and the broker so proceed with caution here.