Details
-
Bug
-
Resolution: Done
-
Major
-
9.4.3.Final
-
None
-
None
Description
I believe that there is a bug in org.infinispan.factories.AbstractComponentRegistry.stop()
Our Wildfly 15 nodes often hang on shutdown on the following:
"MSC service thread 1-2" #37 prio=5 os_prio=0 tid=0x0000000003807000 nid=0xf32d in Object.wait() [0x00007f0012c6f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000005cc795fa8> (a org.infinispan.factories.ComponentRegistry) at java.lang.Object.wait(Object.java:502) at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:359) - locked <0x00000005cc795fa8> (a org.infinispan.factories.ComponentRegistry) at org.infinispan.cache.impl.CacheImpl.performImmediateShutdown(CacheImpl.java:1159) at org.infinispan.cache.impl.CacheImpl.stop(CacheImpl.java:1124) at org.infinispan.cache.impl.AbstractDelegatingCache.stop(AbstractDelegatingCache.java:520) at org.infinispan.manager.DefaultCacheManager.terminate(DefaultCacheManager.java:734) at org.infinispan.manager.DefaultCacheManager.stopCaches(DefaultCacheManager.java:786) at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:762) at org.jboss.as.clustering.infinispan.subsystem.CacheContainerServiceConfigurator.accept(CacheContainerServiceConfigurator.java:114) at org.jboss.as.clustering.infinispan.subsystem.CacheContainerServiceConfigurator.accept(CacheContainerServiceConfigurator.java:70) at org.wildfly.clustering.service.FunctionalService.stop(FunctionalService.java:77) at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:1794) at org.jboss.msc.service.ServiceControllerImpl$StopTask.execute(ServiceControllerImpl.java:1763) at org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1558) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1364) at java.lang.Thread.run(Thread.java:748)
There are no other threads that are working with 0x00000005cc795fa8 in the thread dump.
I checked in a heap dump and the 0x00000005cc795fa8 org.infinispan.factories.ComponentRegistry object state is TERMINATED.
I think that the finally block of AbstractComponentRegistry.stop() is missing a notifyAll().
Shouldn't this:
...
} finally {
synchronized (this) {
state = ComponentStatus.TERMINATED;
}
}
Be:
...
} finally {
synchronized (this) {
state = ComponentStatus.TERMINATED;
notifyAll();
}
}
This way, if two thread try to stop the same cache, the one that wins will notify the one that is waiting letting its stop complete.
