Details
Blocker Description
I'm getting Out of memory when running tests with deployments containing MDB.
Customer Impact: Server can crash on OOME if server with MDB is reloaded.
Steps to reproduce:
- deploy war containing MDB using arquillian
- add jms queue
- reload server <- Above mentioned RejectedExecutionException and AMQ212008 appears during reload.
Server log output:
15:51:23,205 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector-1820600137)) AMQ214013: Failed to decode packet: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask@217b7ea0 rejected from java.util.concurrent.ThreadPoolExecutor@2df83853[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor.execute(OrderedExecutorFactory.java:86) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1309) at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:454) at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:623) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:373) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:356) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1143) at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:196) at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:101) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 15:51:23,869 WARN [org.apache.activemq.artemis.core.client] (Finalizer) AMQ212008: I am closing a core ClientSessionFactory you left open. Please make sure you close all ClientSessionFactories explicitly before letting them go out of scope! 1,620,843,072: java.lang.Exception at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.<init>(ClientSessionFactoryImpl.java:168) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:769) at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation.setup(ActiveMQActivation.java:313) at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation$SetupActivation.run(ActiveMQActivation.java:697) at org.jboss.jca.core.workmanager.WorkWrapper.runWork(WorkWrapper.java:445) at org.jboss.as.connector.services.workmanager.WildflyWorkWrapper.runWork(WildflyWorkWrapper.java:68) at org.jboss.jca.core.workmanager.WorkWrapper.run(WorkWrapper.java:223) at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33) at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:808) at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45) at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:828) at java.lang.Thread.run(Thread.java:745) at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Heap dump from OOM shows following problem:
Finalizer at sun.misc.Unsafe.park(ZJ)V (Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(Ljava/lang/Object;J)V (LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(JLjava/util/concurrent/TimeUnit;)Z (AbstractQueuedSynchronizer.java:2163) at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(Lorg/apache/activemq/artemis/core/protocol/core/Packet;IB)Lorg/apache/activemq/artemis/core/protocol/core/Packet; (ChannelImpl.java:375) at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(Lorg/apache/activemq/artemis/core/protocol/core/Packet;B)Lorg/apache/activemq/artemis/core/protocol/core/Packet; (ChannelImpl.java:315) at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sessionClose()V (ActiveMQSessionContext.java:502) at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.close()V (ClientSessionImpl.java:877) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.closeCleanSessions(Z)V (ClientSessionFactoryImpl.java:442) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.interruptConnectAndCloseAllSessions(Z)V (ClientSessionFactoryImpl.java:424) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.close()V (ClientSessionFactoryImpl.java:457) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.doClose(Z)V (ServerLocatorImpl.java:1364) at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.close()V (ServerLocatorImpl.java:1307) at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.finalize()V (ActiveMQConnectionFactory.java:830) at java.lang.System$2.invokeFinalize(Ljava/lang/Object;)V (System.java:1270) at java.lang.ref.Finalizer.runFinalizer(Lsun/misc/JavaLangAccess;)V (Finalizer.java:98) at java.lang.ref.Finalizer.access$100(Ljava/lang/ref/Finalizer;Lsun/misc/JavaLangAccess;)V (Finalizer.java:34) at java.lang.ref.Finalizer$FinalizerThread.run()V (Finalizer.java:210)
