Details
-
Bug
-
Resolution: Done
-
Major
-
1.0.0.Alpha10
-
None
Description
It is fairly easy to produce a hang in domain mode shutdown by launching domain.sh and then soft killing the process with CTRL-C.
If this hang happens, a thread dump will show this
[Host Controller] "Remoting "localhost:MANAGEMENT" task-16" prio=5 tid=0x00007fa07423b800 nid=0x3b0b waiting for monitor entry [0x000000010bce6000] [Host Controller] java.lang.Thread.State: BLOCKED (on object monitor) [Host Controller] at org.jboss.as.host.controller.ManagedServer.callbackUnregistered(ManagedServer.java:417) [Host Controller] - waiting to lock <0x00000007f7946a30> (a org.jboss.as.host.controller.ManagedServer) [Host Controller] at org.jboss.as.host.controller.ServerInventoryImpl$1.handleClose(ServerInventoryImpl.java:461) [Host Controller] at org.jboss.as.host.controller.ServerInventoryImpl$1.handleClose(ServerInventoryImpl.java:456) [Host Controller] at org.jboss.remoting3.spi.SpiUtils.safeHandleClose(SpiUtils.java:54) [Host Controller] at org.jboss.remoting3.spi.AbstractHandleableCloseable$CloseHandlerTask.run(AbstractHandleableCloseable.java:501) [Host Controller] at org.jboss.remoting3.spi.AbstractHandleableCloseable.runCloseTask(AbstractHandleableCloseable.java:406) [Host Controller] at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeComplete(AbstractHandleableCloseable.java:277) [Host Controller] at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:532) [Host Controller] at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:153) [Host Controller] at org.jboss.as.protocol.mgmt.ManagementChannelReceiver.handleEnd(ManagementChannelReceiver.java:127) [Host Controller] at org.jboss.remoting3.remote.RemoteConnectionChannel$2.run(RemoteConnectionChannel.java:282) [Host Controller] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [Host Controller] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [Host Controller] at java.lang.Thread.run(Thread.java:744) [Host Controller] "Host Controller Service Threads - 13" prio=5 tid=0x00007fa072cbc800 nid=0xa903 waiting on condition [0x000000010d93a000] [Host Controller] java.lang.Thread.State: WAITING (parking) [Host Controller] at sun.misc.Unsafe.park(Native Method) [Host Controller] - parking to wait for <0x00000007f61e5480> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) [Host Controller] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [Host Controller] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) [Host Controller] at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [Host Controller] at org.jboss.as.controller.remote.BlockingQueueOperationListener.retrievePreparedOperation(BlockingQueueOperationListener.java:88) [Host Controller] at org.jboss.as.controller.remote.TransactionalProtocolHandlers.executeBlocking(TransactionalProtocolHandlers.java:76) [Host Controller] at org.jboss.as.host.controller.ManagedServer$ServerStopTask.execute(ManagedServer.java:846) [Host Controller] at org.jboss.as.host.controller.ManagedServer.internalSetState(ManagedServer.java:531) [Host Controller] at org.jboss.as.host.controller.ManagedServer.stop(ManagedServer.java:259) [Host Controller] - locked <0x00000007f7946a30> (a org.jboss.as.host.controller.ManagedServer) [Host Controller] at org.jboss.as.host.controller.ServerInventoryImpl.stopServers(ServerInventoryImpl.java:339) [Host Controller] at org.jboss.as.host.controller.ServerInventoryImpl.shutdown(ServerInventoryImpl.java:437) [Host Controller] at org.jboss.as.host.controller.ServerInventoryService$1.run(ServerInventoryService.java:124) [Host Controller] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [Host Controller] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [Host Controller] at java.lang.Thread.run(Thread.java:744) [Host Controller] at org.jboss.threads.JBossThread.run(JBossThread.java:320)
The 2nd thread is blocking waiting for a prepared response from one of the servers. The first thread is trying to send a notification that the channel connecting to the server has been closed. The first thread will never unblock because the 2nd thread can't proceed and deliver the information that the channel is closed.
This happens because all the processes (PC, HC, servers) respond to the CTRL-C. So the server is shutting down in parallel and has closed the channel.
In theory this could happen for other reasons too, though. Anything that results in closing the channel while the ManagedServer lock is held.
