Uploaded image for project: 'Application Server 7'
  1. Application Server 7
  2. AS7-3145

Server hangs for nine minutes when shutting down with deployments containing EJBs with JMS references

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 7.1.0.Final
    • 7.1.0.CR1b
    • JMS
    • None

    Description

      Build (with Maven 3.0.3) and deploy the attached application.
      Start JBoss 7.1.0.CR1b.
      Stop it with Ctrl-C after startup is complete and the application is deployed.

      After about nine minutes, it stops with:

      21:05:33,682 INFO [org.jboss.weld] Stopping weld service
      21:05:33,682 INFO [org.jboss.as.deployment.connector] JBAS010410: Unbound JCA ConnectionFactory [java:/JmsXA]
      21:05:33,696 INFO [com.arjuna.ats.jbossatx] ARJUNA032018: Destroying TransactionManagerService
      21:05:33,696 INFO [com.arjuna.ats.jbossatx] ARJUNA032014: Stopping transaction recovery manager
      21:05:33,710 INFO [org.jboss.as.server.deployment] Stopped deployment messaging-ejb-1.0-SNAPSHOT.jar in 540189ms
      21:05:33,714 INFO [org.jboss.as.server.deployment] Stopped deployment messaging-web-1.0-SNAPSHOT.war in 540207ms
      21:05:33,716 INFO [org.jboss.as.server.deployment] Stopped deployment messaging-app-1.0-SNAPSHOT.ear in 540195ms
      21:05:33,718 INFO [org.jboss.as] JBoss AS 7.1.0.CR1b "Flux Capacitor" stopped in 540179ms

      This is a thread dump taken during the hang:

      2011-12-29 20:58:09
      Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.4-b02-402 mixed mode):

      "Thread-17 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f1d39800 nid=0x117283000 waiting on condition [117282000]
      java.lang.Thread.State: TIMED_WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d4588510> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-16 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22ec800 nid=0x117180000 in Object.wait() [11717f000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b20698> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b20698> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b20650> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-15 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5eca86800 nid=0x116ac2000 in Object.wait() [116ac1000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b1e058> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b1e058> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b1e010> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-14 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f082b800 nid=0x116856000 in Object.wait() [116855000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b21358> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b21358> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b21310> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-13 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5ecb96000 nid=0x116753000 in Object.wait() [116752000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b28e20> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b28e20> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b28dd8> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-11 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5eca31800 nid=0x116427000 in Object.wait() [116426000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b2c120> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b2c120> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b2c0d8> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-10 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5ec94c000 nid=0x116324000 in Object.wait() [116323000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b1ed18> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b1ed18> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b1ecd0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-9 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22eb800 nid=0x116221000 in Object.wait() [116220000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b29ae0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b29ae0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b29a98> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-8 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22eb000 nid=0x115ffc000 in Object.wait() [115ffb000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b1d398> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b1d398> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b1d350> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-7 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22ea000 nid=0x115ef9000 in Object.wait() [115ef8000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b14a80> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b14a80> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b14a20> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-6 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22e9800 nid=0x115df6000 in Object.wait() [115df5000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b1f9d8> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b1f9d8> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b1f990> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-5 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22e8800 nid=0x115c27000 in Object.wait() [115c26000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b274a0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b274a0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b27458> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-4 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5eca2a800 nid=0x115b24000 in Object.wait() [115b23000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5af6070> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5af6070> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5af6028> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-3 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f22b6800 nid=0x11591b000 in Object.wait() [11591a000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b2a7a0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b2a7a0> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b2a758> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-2 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f2229000 nid=0x114f13000 in Object.wait() [114f12000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b28160> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b28160> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b28118> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "MSC Shutdown Thread" prio=5 tid=7fc5f1d35000 nid=0x115818000 waiting on condition [115817000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d05dba10> (a java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
        at org.jboss.msc.service.ServiceContainerImpl$ShutdownHookHolder$1$1.run(ServiceContainerImpl.java:129)
        at java.lang.Thread.run(Thread.java:680)

      "SIGINT handler" daemon prio=9 tid=7fc5f0b23800 nid=0x115715000 in Object.wait() [115714000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d41e5d98> (a java.lang.Thread)
        at java.lang.Thread.join(Thread.java:1210)
      • locked <7d41e5d98> (a java.lang.Thread)
        at java.lang.Thread.join(Thread.java:1263)
        at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
        at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
        at java.lang.Shutdown.runHooks(Shutdown.java:79)
        at java.lang.Shutdown.sequence(Shutdown.java:123)
        at java.lang.Shutdown.exit(Shutdown.java:168)
      • locked <7f019d288> (a java.lang.Class for java.lang.Shutdown)
        at java.lang.Terminator$1.handle(Terminator.java:35)
        at sun.misc.Signal$1.run(Signal.java:195)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-1 (group:HornetQ-client-global-threads-310083381)" daemon prio=5 tid=7fc5f0801800 nid=0x11ccdb000 in Object.wait() [11ccda000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d5b2b460> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:916)
      • locked <7d5b2b460> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:840)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:588)
      • locked <7d5b2b418> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl.access$800(ClientSessionFactoryImpl.java:78)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1318)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:528)
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:298)
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler$1.run(ClientSessionFactoryImpl.java:1262)
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-4 (group:HornetQ-client-global-scheduled-threads-164700615)" daemon prio=5 tid=7fc5f11c6000 nid=0x11cbd8000 waiting on condition [11cbd7000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5b147f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-3 (group:HornetQ-client-global-scheduled-threads-164700615)" daemon prio=5 tid=7fc5f11c5000 nid=0x11c6c9000 waiting on condition [11c6c8000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5b147f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-2 (group:HornetQ-client-global-scheduled-threads-164700615)" daemon prio=5 tid=7fc5f093f800 nid=0x11c4c3000 waiting on condition [11c4c2000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5b147f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-1 (group:HornetQ-client-global-scheduled-threads-164700615)" daemon prio=5 tid=7fc5f13f6000 nid=0x11bfb4000 waiting on condition [11bfb3000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5b147f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Thread-0 (group:HornetQ-client-global-scheduled-threads-164700615)" daemon prio=5 tid=7fc5ee43f000 nid=0x11a5c6000 waiting on condition [11a5c5000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5b147f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Transaction Reaper Worker 0" daemon prio=5 tid=7fc5ee178800 nid=0x11b390000 in Object.wait() [11b38f000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d509b2c0> (a java.util.LinkedList)
        at java.lang.Object.wait(Object.java:485)
        at com.arjuna.ats.arjuna.coordinator.TransactionReaper.waitForCancellations(TransactionReaper.java:318)
      • locked <7d509b2c0> (a java.util.LinkedList)
        at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:66)

      "Transaction Reaper" daemon prio=5 tid=7fc5ee177800 nid=0x11b28d000 in Object.wait() [11b28c000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4eab738> (a com.arjuna.ats.arjuna.coordinator.TransactionReaper)
        at com.arjuna.ats.internal.arjuna.coordinator.ReaperThread.run(ReaperThread.java:91)
      • locked <7d4eab738> (a com.arjuna.ats.arjuna.coordinator.TransactionReaper)

      "AOMRemoval" daemon prio=5 tid=7fc5eccac800 nid=0x11a97f000 waiting on condition [11a97e000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d509b3b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at org.jacorb.poa.AOM$1.run(AOM.java:120)

      "AOMRemoval" daemon prio=5 tid=7fc5ee1f1800 nid=0x119cba000 waiting on condition [119cb9000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5093b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at org.jacorb.poa.AOM$1.run(AOM.java:120)

      "AOMRemoval" daemon prio=5 tid=7fc5ecb11800 nid=0x11ad7e000 waiting on condition [11ad7d000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5093b00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at org.jacorb.poa.AOM$1.run(AOM.java:120)

      "Periodic Recovery" prio=5 tid=7fc5ee26f800 nid=0x11ab75000 in Object.wait() [11ab74000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d509b410> (a java.lang.Object)
        at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doPeriodicWait(PeriodicRecovery.java:672)
        at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:392)
      • locked <7d509b410> (a java.lang.Object)

      "Transaction Expired Entry Monitor" daemon prio=5 tid=7fc5f0226000 nid=0x11a779000 in Object.wait() [11a778000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4f7b240> (a com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor)
        at com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor.run(ExpiredEntryMonitor.java:191)
      • locked <7d4f7b240> (a com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor)

      "http-localhost-127.0.0.1-8080-Acceptor" daemon prio=5 tid=7fc5f0014800 nid=0x119bad000 waiting on condition [119bac000]
      java.lang.Thread.State: TIMED_WAITING (sleeping)
      at java.lang.Thread.sleep(Native Method)
      at org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:1065)
      at java.lang.Thread.run(Thread.java:680)

      "http-localhost-127.0.0.1-8080-Sendfile" daemon prio=5 tid=7fc5f0225000 nid=0x119aa8000 in Object.wait() [119aa7000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4a50960> (a org.apache.tomcat.util.net.AprEndpoint$Sendfile)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Sendfile.run(AprEndpoint.java:2258)
      • locked <7d4a50960> (a org.apache.tomcat.util.net.AprEndpoint$Sendfile)
        at java.lang.Thread.run(Thread.java:680)

      "http-localhost-127.0.0.1-8080-EventPoller" daemon prio=5 tid=7fc5f009b000 nid=0x1199a3000 in Object.wait() [1199a2000]
      java.lang.Thread.State: TIMED_WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4ecdb50> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1678)
      • locked <7d4ecdb50> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Thread.run(Thread.java:680)

      "Timer-0" prio=5 tid=7fc5f0927000 nid=0x1195df000 in Object.wait() [1195de000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4a38cb0> (a java.util.TaskQueue)
        at java.lang.Object.wait(Object.java:485)
        at java.util.TimerThread.mainLoop(Timer.java:483)
      • locked <7d4a38cb0> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

      "DestroyJavaVM" prio=5 tid=7fc5f00df000 nid=0x10ba5e000 waiting on condition [00000000]
      java.lang.Thread.State: RUNNABLE

      "MSC service thread 1-8" prio=5 tid=7fc5ee0d5800 nid=0x114ddf000 waiting on condition [114dde000]
      java.lang.Thread.State: TIMED_WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d5b2bf10> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2116)
        at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:256)
      • locked <7d5b2bf58> (a java.lang.Object)
        at org.hornetq.core.client.impl.ClientSessionImpl.close(ClientSessionImpl.java:857)
        at org.hornetq.core.client.impl.DelegatingSession.close(DelegatingSession.java:152)
        at org.hornetq.ra.inflow.HornetQMessageHandler.teardown(HornetQMessageHandler.java:242)
        at org.hornetq.ra.inflow.HornetQActivation.teardown(HornetQActivation.java:319)
      • locked <7d5aae0a0> (a org.hornetq.ra.inflow.HornetQActivation)
        at org.hornetq.ra.inflow.HornetQActivation.stop(HornetQActivation.java:269)
        at org.hornetq.ra.HornetQResourceAdapter.endpointDeactivation(HornetQResourceAdapter.java:175)
        at org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponent.stop(MessageDrivenComponent.java:167)
        at org.jboss.as.ee.component.ComponentStartService.stop(ComponentStartService.java:49)
        at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1909)
        at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:1872)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-7" prio=5 tid=7fc5f0863800 nid=0x114cdc000 waiting on condition [114cdb000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-6" prio=5 tid=7fc5ecbba000 nid=0x114bd9000 waiting on condition [114bd8000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-5" prio=5 tid=7fc5ecbb9800 nid=0x114ad6000 waiting on condition [114ad5000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Poller SunPKCS11-Darwin" daemon prio=1 tid=7fc5ecbb8800 nid=0x1149d3000 waiting on condition [1149d2000]
      java.lang.Thread.State: TIMED_WAITING (sleeping)
      at java.lang.Thread.sleep(Native Method)
      at sun.security.pkcs11.SunPKCS11$TokenPoller.run(SunPKCS11.java:692)
      at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-4" prio=5 tid=7fc5ecbb4000 nid=0x1148c3000 waiting on condition [1148c2000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-3" prio=5 tid=7fc5ee0d4800 nid=0x1147c0000 waiting on condition [1147bf000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-2" prio=5 tid=7fc5f08d7800 nid=0x1146bd000 waiting on condition [1146bc000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "MSC service thread 1-1" prio=5 tid=7fc5ee228000 nid=0x1145ba000 waiting on condition [1145b9000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <7d40292e8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:680)

      "Reference Reaper" daemon prio=5 tid=7fc5ee098800 nid=0x11403e000 in Object.wait() [11403d000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4040d00> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
      • locked <7d4040d00> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at org.jboss.modules.ref.References$ReaperThread.run(References.java:68)

      "Low Memory Detector" daemon prio=5 tid=7fc5ee007000 nid=0x113be7000 runnable [00000000]
      java.lang.Thread.State: RUNNABLE

      "C2 CompilerThread1" daemon prio=9 tid=7fc5ee006000 nid=0x113ae4000 waiting on condition [00000000]
      java.lang.Thread.State: RUNNABLE

      "C2 CompilerThread0" daemon prio=9 tid=7fc5ee005800 nid=0x1139e1000 waiting on condition [00000000]
      java.lang.Thread.State: RUNNABLE

      "Signal Dispatcher" daemon prio=9 tid=7fc5ee004800 nid=0x1138de000 waiting on condition [00000000]
      java.lang.Thread.State: RUNNABLE

      "Surrogate Locker Thread (Concurrent GC)" daemon prio=5 tid=7fc5ee003800 nid=0x1137db000 waiting on condition [00000000]
      java.lang.Thread.State: RUNNABLE

      "Finalizer" daemon prio=8 tid=7fc5ee001000 nid=0x113517000 in Object.wait() [113516000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d40886e8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
      • locked <7d40886e8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

      "Reference Handler" daemon prio=10 tid=7fc5ee000000 nid=0x113414000 in Object.wait() [113413000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <7d4021ae0> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
      • locked <7d4021ae0> (a java.lang.ref.Reference$Lock)

      "VM Thread" prio=9 tid=7fc5ec8eb800 nid=0x113311000 runnable

      "Gang worker#0 (Parallel GC Threads)" prio=9 tid=7fc5ec802800 nid=0x10ed99000 runnable

      "Gang worker#1 (Parallel GC Threads)" prio=9 tid=7fc5ec803800 nid=0x10ee9c000 runnable

      "Gang worker#2 (Parallel GC Threads)" prio=9 tid=7fc5ec804000 nid=0x10ef9f000 runnable

      "Gang worker#3 (Parallel GC Threads)" prio=9 tid=7fc5ec804800 nid=0x10f0a2000 runnable

      "Concurrent Mark-Sweep GC Thread" prio=9 tid=7fc5ec87f800 nid=0x112fb7000 runnable
      "VM Periodic Task Thread" prio=10 tid=7fc5f0800000 nid=0x113cea000 waiting on condition

      "Exception Catcher Thread" prio=10 tid=7fc5ec802000 nid=0x10bbc0000 runnable
      JNI global references: 1552

      Heap
      par new generation total 19136K, used 13050K [7d0000000, 7d14c0000, 7d4000000)
      eden space 17024K, 65% used [7d0000000, 7d0ae7d60, 7d10a0000)
      from space 2112K, 89% used [7d10a0000, 7d1276e88, 7d12b0000)
      to space 2112K, 0% used [7d12b0000, 7d12b0000, 7d14c0000)
      concurrent mark-sweep generation total 44288K, used 28846K [7d4000000, 7d6b40000, 7f0000000)
      concurrent-mark-sweep perm gen total 109064K, used 69234K [7f0000000, 7f6a82000, 800000000)

      Attachments

        Issue Links

          is related to

          Bug - A problem that impairs or prevents the functions of the product. AS7-3361 HornetQ failover hangs

          • Critical - To be worked on immediately following BLOCKER issues.
          • Resolved

          Bug - A problem that impairs or prevents the functions of the product. AS7-3563 java.lang.OutOfMemoryError: unable to create new native thread

          • Critical - To be worked on immediately following BLOCKER issues.
          • Resolved

          Activity

            People

              rh-ee-ataylor Andy Taylor
              sfcoy Stephen Coy (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: