Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-9701

TransactionTable does not shutdown gracefully

    XMLWordPrintable

Details

    Description

      Here's a sample stacktrace during shutdown:

      16:54:15,033 WARN  [org.wildfly.clustering.web.undertow] (default task-1) ISPN000472: Cache manager is stopping: org.infinispan.IllegalLifecycleStateException: ISPN000472: Cache manager is stopping
	at org.infinispan.marshall.core.GlobalMarshaller.getExternalizer(GlobalMarshaller.java:420)
	at org.infinispan.marshall.core.GlobalMarshaller.writeNonNullableObject(GlobalMarshaller.java:400)
	at org.infinispan.marshall.core.GlobalMarshaller.writeNullableObject(GlobalMarshaller.java:355)
	at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:183)
	at org.infinispan.marshall.core.GlobalMarshaller.writeObjectOutput(GlobalMarshaller.java:176)
	at org.infinispan.marshall.core.GlobalMarshaller.objectToBuffer(GlobalMarshaller.java:305)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.marshallRequest(JGroupsTransport.java:1009)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.sendCommand(JGroupsTransport.java:1209)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.performAsyncRemoteInvocation(JGroupsTransport.java:1105)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotelyAsync(JGroupsTransport.java:246)
	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotelyAsync(RpcManagerImpl.java:291)
	at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:323)
	at org.infinispan.transaction.impl.TransactionTable.removeTransactionInfoRemotely(TransactionTable.java:900)
	at org.infinispan.transaction.impl.TransactionTable.releaseLocksForCompletedTransaction(TransactionTable.java:886)
	at org.infinispan.transaction.xa.XaTransactionTable.forgetSuccessfullyCompletedTransaction(XaTransactionTable.java:195)
	at org.infinispan.transaction.xa.XaTransactionTable.commit(XaTransactionTable.java:128)
	at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:68)
	at org.infinispan.commons.tx.TransactionImpl.finishResource(TransactionImpl.java:419)
	at org.infinispan.commons.tx.TransactionImpl.commitResources(TransactionImpl.java:466)
	at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:335)
	at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110)

      The problem seems to be that shutDownGracefully() first waits for the localTransactions map to be empty. However, when the cache is clustered, releaseLocksForCompletedTransaction(...) removes the transaction from the localTransactions map before invoking removeTransactionInfoRemotely(...), which means that the subsequent TxCompletionNotificationCommand can fail to marshal (see above), or the transport might close before this command is sent.

      A naive fix would simply reorder the removeLocalTransaction(...) to happen after the call to removeTransactionInfoRemotely(...) within the releaseLocksForCompletedTransaction(...) method, but I'm sure there's more to it.

      Attachments

        Issue Links

          causes

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-15906 (7.2.z) WARN HeuristicMixedException due to replication timeout

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. WFLY-11322 RemoteStatefulEJBConcurrentFailoverTestCase is unreliable

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. WFLY-11384 StatefulFailoverTestCase fails intermittently

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. WFLY-11389 JSFFailoverTestCase fails intermittently

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-16218 (7.2.z) ISPN-9701 - TransactionTable does not shutdown gracefully

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified
          is incorporated by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-11401 Upgrade Infinispan to 9.4.3.Final

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-11413 Upgrade Infinispan to 9.4.3.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              pferraro@redhat.com Paul Ferraro
              pferraro@redhat.com Paul Ferraro
              Diego Lovison Diego Lovison
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: