Details
-
Bug
-
Resolution: Cannot Reproduce
-
Critical
-
None
-
5.1.4.FINAL
-
None
Description
During elasticity test with this timeline:
start node1, REPLICATED clustering mode
wait for start
load data
start node2
wait for cluster to form
start node3
wait for cluster to form
...
node3 fails to join the cluster with the following stack trace:
node03: 06:58:06,731 ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-3,node03/default) ISPN000172: Failed to prepare view CacheView
on cache ___hotRodTopologyCache, we are currently preparing view CacheView
{viewId=3, members=[node01/default, node02/default, node03/default]}node03: at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_30]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:321) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:249) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:875) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
node03: at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
node03: Caused by: java.lang.IllegalStateException: Cannot prepare new view CacheView{viewId=1, members=[node03/default]} on cache ___hotRodTopologyCache, we are currently preparing view CacheView{viewId=3, members=[node01/default, node02/default, node03/default]}
node03: at org.infinispan.cacheviews.CacheViewInfo.prepareView(CacheViewInfo.java:102) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:481) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl$3.call(CacheViewsManagerImpl.java:312) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
node03: at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
node03: at org.jboss.threads.JBossThread.run(JBossThread.java:122)
node03:
node03: 06:58:07,728 ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-3,node03/default) ISPN000172: Failed to prepare view CacheView
for cache ___hotRodTopologyCache, rolling back to view CacheView
{viewId=2, members=[]}: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Trying to block write commands but they are already blocked for view 3
node03: at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_30]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:321) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:249) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:875) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
node03: at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
node03: Caused by: java.lang.IllegalStateException: Trying to block write commands but they are already blocked for view 3
node03: at org.infinispan.statetransfer.StateTransferLockImpl.blockNewTransactions(StateTransferLockImpl.java:233) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.statetransfer.ReplicatedStateTransferTask.doPerformStateTransfer(ReplicatedStateTransferTask.java:84) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.statetransfer.BaseStateTransferTask.performStateTransfer(BaseStateTransferTask.java:93) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.statetransfer.BaseStateTransferManagerImpl.prepareView(BaseStateTransferManagerImpl.java:331) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:485) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at org.infinispan.cacheviews.CacheViewsManagerImpl$3.call(CacheViewsManagerImpl.java:312) [infinispan-core-5.1.5.ER8-redhat-1.jar:5.1.5.ER8-redhat-1]
node03: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_30]
node03: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_30]
node03: at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_30]
node03: at org.jboss.threads.JBossThread.run(JBossThread.java:122)
node03:
node03: 07:19:37,679 ERROR [org.infinispan.cacheviews.CacheViewInfo] (undefined) ISPN000164: Rolling back to cache view 2, but last committed view is 1
node03: 07:19:37,687 ERROR [org.infinispan.cacheviews.CacheViewInfo] (undefined) ISPN000164: Rolling back to cache view 5, but last committed view is 1
node03: 07:19:37,689 ERROR [org.infinispan.cacheviews.CacheViewInfo] (undefined) ISPN000164: Rolling back to cache view 6, but last committed view is 1
node03: 07:19:47,684 ERROR [org.infinispan.cacheviews.CacheViewInfo] (undefined) ISPN000164: Rolling back to cache view 3, but last committed view is 5