Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-3107

java.lang.OutOfMemoryError: Direct buffer memory

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • AMQ 7.4.2.GA
    • AMQ 7.4.0.GA
    • amqp-protocol
    • None

    Description

      AMQ 7.4,
      -Using this test client: https://github.com/rh-ei-stp/fuse-client-app/tree/no-prefetch-test will get a Direct Buffer OOM after some time of execution.
      -ENTMQBR-2599, suggests add -Djdk.nio.maxCachedBufferSize=100000 (or less) to the broker JVM startup parameters while using large messages with OpenWire/AMQP to prevent NIO from caching large (> 100 KB) direct ByteBuffers, but it just delay the crash condition, maybe even for days.
      -Sample stacktrace:

      2019-08-05 07:00:06,511 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: removing consumer which did not handle a message, consumer=3DServerConsumerImpl [id=3D0, filter=3Dnull, binding=3DLocalQueueBinding [address=3DPHLCA.SYNCPIX.OUT, queue=3DQueueImpl[name=3DPHLCA.SYNCPIX.OUT, postOffice=3DPostOfficeImpl [server=3DActiveMQServerImpl::serverUUID=3D36d7573b-b53f-11e9-b3a3-0200170151fc], temp=3Dfalse]@3f70a55c, filter=3Dnull, name=3DPHLCA.SYNCPIX.OUT, clusterName=3DPHLCA.SYNCPIX.OUT36d7573b-b53f-11e9-b3a3-0200170151fc]], message=3DReference[594177]:RELIABLE:LargeServerMessage[messageID=3D594177,durable=3Dtrue,userID=3Dnull,priority=3D4, timestamp=3DMon Aug 05 07:00:04 EDT 2019,expiration=3D0, durable=3Dtrue, address=3DPHLCA.SYNCPIX.OUT, properties=3DTypedProperties[__HDR_BROKER_IN_TIME=3D1565002804443,_AMQ_ROUTING_TYPE=3D1,__HDR_GROUP_SEQUENCE=3D0,__HDR_COMMAND_ID=3D11409,_AMQ_LARGE_SIZE=3D5331599,__HDR_MESSAGE_ID=3D[0000 007E 6E00 017B 0100 5749 443A 6673 6370 7068 6C63 6177 6562 3032 2E61  ...  0000 0000 0A4D 0000 0000 0000 0002 0000 0000 0000 0001 0000 0000 0000 0000),__HDR_DROPPABLE=3Dfalse,__AMQ_CID=3DID:fscpphlcaweb02.a10199210024oci.a10199210023oci.oraclevcn.com-45393-1564435941942-3:4,MOD_ID=3D1627053435,__HDR_ARRIVAL=3D0,__HDR_MARSHALL_PROP=3D[0000 0002 000B 4F52 4947 5F53 5953 5445 4D09 0005 776D 7765 6200 064D 4F44 5F49 4405 60FA DD7B),ORIG_SYSTEM=3Dwmweb,__HDR_PRODUCER_ID=3D[0000 006B 7B01 0057 4944 3A66 7363 7070 686C 6361 7765 6230 322E 6131 3031  ... 34 3433 3539 3431 3934 322D 343A 3400 0000 0000 000A 4D00 0000 0000 0000 02)]]@604055317: java.lang.OutOfMemoryError: Direct buffer memory
        at java.nio.Bits.reserveMemory(Bits.java:695) [rt.jar:1.8.0_191]
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) [rt.jar:1.8.0_191]
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) [rt.jar:1.8.0_191]
        at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241) [rt.jar:1.8.0_191]
        at sun.nio.ch.IOUtil.read(IOUtil.java:195) [rt.jar:1.8.0_191]
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:159) [rt.jar:1.8.0_191]
        at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.read(NIOSequentialFile.java:165) [artemis-journal-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.read(NIOSequentialFile.java:155) [artemis-journal-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl$DecodingContext.encode(LargeServerMessageImpl.java:538) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.message.impl.CoreMessage.getLargeMessageBuffer(CoreMessage.java:262) [artemis-core-client-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.message.impl.CoreMessage.getDataBuffer(CoreMessage.java:239) [artemis-core-client-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.protocol.amqp.converter.jms.ServerJMSMessage.getReadBodyBuffer(ServerJMSMessage.java:76) [artemis-amqp-protocol-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.protocol.amqp.converter.jms.ServerJMSTextMessage.decode(ServerJMSTextMessage.java:94) [artemis-amqp-protocol-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.protocol.amqp.converter.CoreAmqpConverter.fromCore(CoreAmqpConverter.java:125) [artemis-amqp-protocol-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.protocol.amqp.converter.CoreAmqpConverter.checkAMQP(CoreAmqpConverter.java:109) [artemis-amqp-protocol-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.protocol.amqp.proton.ProtonServerSenderContext.deliverMessage(ProtonServerSenderContext.java:671) [artemis-amqp-protocol-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.protocol.amqp.broker.AMQPSessionCallback.sendMessage(AMQPSessionCallback.java:636) [artemis-amqp-protocol-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.deliverStandardMessage(ServerConsumerImpl.java:1107) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:465) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2942) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2410) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2000(QueueImpl.java:107) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:3216) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191]
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]

      Attachments

        Issue Links

          clones

          Bug - A problem that impairs or prevents the functions of the product. ENTMQBR-2853 java.lang.OutOfMemoryError: Direct buffer memory

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Activity

            People

              fnigro Francesco Nigro
              dbruscin Domenico Francesco Bruscino
              Tiago Bueno Tiago Bueno
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: