Uploaded image for project: 'JBoss A-MQ'
  1. JBoss A-MQ
  2. ENTMQ-2215

[AMQ 6, memory, pendingCachedIds ] org.apache.activemq.broker.region.cursors.QueueStorePrefetch.pendingCachedIds grows indefinitely

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • JBoss A-MQ 6.3.x
    • JBoss A-MQ 6.3
    • broker
    • None
    • +
    • Hide

      TODO

      Show
      TODO

    Description

      After the following exception is experienced on the broker, the org.apache.activemq.broker.region.cursors.QueueStorePrefetch.pendingCachedIds list seems to grow indefinitely causing a memory leak.

      javax.jms.JMSException: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
      	at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
      	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1398)
      	at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1324)
      	at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1905)
      	at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:288)
      	at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:223)
      	at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
      	at(appcode)
      Caused by: java.lang.Throwable: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
      	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
      	at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:881)
      	at org.apache.activemq.broker.region.Queue.send(Queue.java:752)
      	at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:132)
      	at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:432)
      	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:468)
      	at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:296)
      	at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
      	at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:293)
      	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:153)
      	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:153)
      	at org.apache.activemq.security.AuthorizationBroker.send(AuthorizationBroker.java:226)
      	at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:158)
      	at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:571)
      	at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:768)
      	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:326)
      	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:190)
      	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
      	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:125)
      	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:301)
      	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
      	at org.apache.activemq.transport.nio.NIOTransport.serviceRead(NIOTransport.java:141)
      	at org.apache.activemq.transport.nio.NIOTransport$1.onSelect(NIOTransport.java:70)
      	at org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:98)
      	at org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:118)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.util.ConcurrentModificationException
      	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1442)
      	at java.util.HashMap$KeyIterator.next(HashMap.java:1466)
      	at org.apache.activemq.util.MarshallingSupport.marshalPrimitiveMap(MarshallingSupport.java:61)
      	at org.apache.activemq.command.Message.beforeMarshall(Message.java:240)
      	at org.apache.activemq.openwire.v6.MessageMarshaller.looseMarshal(MessageMarshaller.java:277)
      	at org.apache.activemq.openwire.v6.ActiveMQMessageMarshaller.looseMarshal(ActiveMQMessageMarshaller.java:111)
      	at org.apache.activemq.openwire.v6.ActiveMQBytesMessageMarshaller.looseMarshal(ActiveMQBytesMessageMarshaller.java:111)
      	at org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:156)
      	at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.addMessage(KahaDBStore.java:435)
      	at org.apache.activemq.store.kahadb.KahaDBStore$StoreQueueTask.run(KahaDBStore.java:1307)
      	... 3 more
      

      It seems once this exception is thrown, the org.apache.activemq.broker.region.cursors.AbstractStoreCursor#pruneLastCached() operation will not traverse the list as the first item in the List is a Future and it is not cancelled.

      Attachments

        Issue Links

          Activity

            People

              gtully@redhat.com Gary Tully
              rhn-support-pfox Patrick Fox (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: