Details
-
Bug
-
Resolution: Done
-
Major
-
9.1.0.Final
-
None
Description
The issue seems to be that during several topology changes the originator sends 2 VersionedPrepareCommands, 1 VersionedCommitCommand and 1 TxCompletionNotificationCommand. These commands are resent after cluster merge and then executed in arbitrary order, and after the completion notification command removes tx information from TransactionTable another prepare command adds it again.
Logs attached, don't get confused by the fact that
14:54:29,056 TRACE [org.infinispan.transaction.impl.TransactionTable] (remote-thread-test-NodeC-p146-t4) Created and registered remote transaction RemoteTransaction{modifications=[PutKeyValueCommand{key=MagicKey#k1{8/537A1109/107@test-NodeB-18592}, value=final-value, flags=[], commandInvocationId=CommandInvocation:local:0, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=13}, PutKeyValueCommand{key=MagicKey#k2{9/0AD74A7B/101@test-NodeC-10992}, value=final-value, flags=[], commandInvocationId=CommandInvocation:local:0, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=13}], lookedUpEntries={}, lockedKeys=[], backupKeyLocks=[], lookedUpEntriesTopology=2147483647, isMarkedForRollback=false, tx=GlobalTx:test-NodeA-55522:15, state=null}
appears in the log before
14:54:29,056 TRACE [org.infinispan.transaction.impl.TransactionTable] (remote-thread-test-NodeC-p146-t6) Removed remote transaction GlobalTx:test-NodeA-55522:15 ? RemoteTransaction{modifications=[PutKeyValueComm
and{key=MagicKey#k1{8/537A1109/107@test-NodeB-18592}, value=final-value, flags=[], commandInvocationId=CommandInvocation:local:0, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{
lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=13}, PutKeyValueCommand{key=MagicKey#k2{9/0AD74A7B/101@test-NodeC-10992}, value=final-value, flags=[], commandInvocationId=CommandInvocation:lo
cal:0, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=null}, successful=true, topologyId=13}], lookedUpEntries={MagicKey#k2{9/0AD74A7B/101@test-
NodeC-10992}=VersionedRepeatableReadEntry(51ecb641){key=MagicKey#k2{9/0AD74A7B/101@test-NodeC-10992}, value=final-value, isCreated=false, isChanged=true, isRemoved=false, isExpired=false, skipLookup=true, metada
ta=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=SimpleClusteredVersion{topologyId=13, version=1}}}, MagicKey#k1{8/537A1109/107@test-NodeB-18592}=VersionedRepeatableReadEntry(7a4c1b3f){key=MagicKey#
k1{8/537A1109/107@test-NodeB-18592}, value=final-value, isCreated=false, isChanged=true, isRemoved=false, isExpired=false, skipLookup=true, metadata=EmbeddedExpirableMetadata{lifespan=-1, maxIdle=-1, version=Sim
pleClusteredVersion{topologyId=13, version=1}}}}, lockedKeys=[MagicKey#k2{9/0AD74A7B/101@test-NodeC-10992}], backupKeyLocks=[MagicKey#k1{8/537A1109/107@test-NodeB-18592}], lookedUpEntriesTopology=14, isMarkedFor
Rollback=false, tx=GlobalTx:test-NodeA-55522:15, state=null}
While we check TransactionTable.isTransactionCompleted() within remoteTransactions.compute, there is a race between calling TransactionTable.removeRemoteTransaction and TransactionTable.markTransactionCompleted. My suggestion is to mark transaction as complete from within the removeRemoteTransaction method, using remoteTransactions.compute.
