Details
-
Bug
-
Resolution: Done
-
Major
-
5.1.0.FINAL
-
None
Description
When a node leaves the cluster, it remains in the list of requestors for all the entries that it retrieved from other nodes. When those entries are updated, their owners try to invalidate them on the leaver. The invalidation command fails because the target has left, and this causes the entire put operation to fail.
This can be seen in some of the HotRod server tests, e.g. HotRodDistributionTest:
2012-02-02 10:56:06,186 TRACE (CacheViewTrigger,HotRodDistributionTest-NodeA-37471) [org.infinispan.cacheviews.PendingCacheViewChanges] Previous members are [HotRodDistributionTest-NodeA-37471, HotRodDistributionTest-NodeB-63614, HotRodDistributionTest-NodeC-41658], joiners are [], leavers are [HotRodDistributionTest-NodeC-41658], recovered after merge = false
2012-02-02 10:56:06,296 TRACE (OOB-2,ISPN,HotRodDistributionTest-NodeB-63614) [org.infinispan.distribution.L1ManagerImpl] Keys [ByteArrayKey{data=ByteArray{size=39, hashCode=7ec14a15, array=0x6b2d746573744469..}}] needs invalidation on [HotRodDistributionTest-NodeC-41658]
2012-02-02 10:56:06,296 TRACE (OOB-2,ISPN,HotRodDistributionTest-NodeB-63614) [org.infinispan.remoting.rpc.RpcManagerImpl] HotRodDistributionTest-NodeB-63614 invoking in future call InvalidateL1Command{num keys=1, forRehash=false, origin=HotRodDistributionTest-NodeA-37471} to recipient list [HotRodDistributionTest-NodeC-41658]
2012-02-02 10:56:06,297 TRACE (transport-thread-64) [org.infinispan.remoting.rpc.RpcManagerImpl] HotRodDistributionTest-NodeB-63614 broadcasting call InvalidateL1Command{num keys=1, forRehash=false, origin=HotRodDistributionTest-NodeA-37471} to recipient list [HotRodDistributionTest-NodeC-41658]
2012-02-02 10:56:06,298 ERROR (OOB-2,ISPN,HotRodDistributionTest-NodeB-63614) [org.infinispan.interceptors.InvocationContextInterceptor] ISPN000136: Execution error
java.util.concurrent.ExecutionException: org.infinispan.remoting.transport.jgroups.SuspectException: One or more nodes have left the cluster while replicating command SingleRpcCommand{cacheName='hotRodDistSync', command=InvalidateL1Command{num keys=1, forRehash=false, origin=HotRodDistributionTest-NodeA-37471}}