Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-6282

Exceptions in 2-clusters EJB invocation graceful shutdown tests

XMLWordPrintable

      I have tests that start 2 clusters and a standalone EJB client and do remote EJB invocations like this: standalone EJB client -> cluster 1 (forwarder) -> cluster 2 (target) -> back to cluster 1 -> back to standalone client.

      During a graceful shutdown variant of such tests, I've seen two unexpected exceptions.


      This one happens during the time the graceful shutdown is initiated. I remember that transactions don't yet support graceful shutdown, so maybe that's the cause? There's an IOException: Channel ... of Remoting connection ... to localhost/127.0.0.1:8280 has been closed down there, so maybe my guess about graceful shutdown is wrong?

      javax.ejb.EJBException: javax.transaction.HeuristicMixedException
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:138)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:119)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:79)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:53)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
      	at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
      	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.invokeMethod(MethodInvocationMessageHandler.java:327)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.access$100(MethodInvocationMessageHandler.java:67)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler$1.run(MethodInvocationMessageHandler.java:200)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.processMessage(MethodInvocationMessageHandler.java:262)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.processMessage(VersionOneProtocolChannelReceiver.java:213)
      	at org.jboss.as.ejb3.remote.protocol.versiontwo.VersionTwoProtocolChannelReceiver.processMessage(VersionTwoProtocolChannelReceiver.java:76)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.handleMessage(VersionOneProtocolChannelReceiver.java:159)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:456)
      	at org.jboss.remoting3.EndpointImpl$TrackingExecutor$1.run(EndpointImpl.java:717)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      	at ...asynchronous invocation...(Unknown Source)
      	at org.jboss.ejb.client.remoting.InvocationExceptionResponseHandler$MethodInvocationExceptionResultProducer.getResult(InvocationExceptionResponseHandler.java:96)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:276)
      	at org.jboss.ejb.client.EJBObjectInterceptor.handleInvocationResult(EJBObjectInterceptor.java:64)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290)
      	at org.jboss.ejb.client.EJBHomeInterceptor.handleInvocationResult(EJBHomeInterceptor.java:88)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290)
      	at org.jboss.ejb.client.TransactionInterceptor.handleInvocationResult(TransactionInterceptor.java:46)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290)
      	at org.jboss.ejb.client.ReceiverInterceptor.handleInvocationResult(ReceiverInterceptor.java:142)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:265)
      	at org.jboss.ejb.client.EJBClientInvocationContext.awaitResponse(EJBClientInvocationContext.java:453)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:204)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)
      	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
      	at com.sun.proxy.$Proxy26.test(Unknown Source)
      	at org.jboss.qa.eap.clustering.[redacted].tools.ClientThread.doCall(ClientThread.java:75)
      	at org.jboss.qa.eap.clustering.[redacted].tools.ClientThread.run(ClientThread.java:56)
      Caused by: javax.transaction.HeuristicMixedException
      	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1208)
      	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
      	at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:79)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:53)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
      	at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
      	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.invokeMethod(MethodInvocationMessageHandler.java:327)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.access$100(MethodInvocationMessageHandler.java:67)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler$1.run(MethodInvocationMessageHandler.java:200)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.processMessage(MethodInvocationMessageHandler.java:262)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.processMessage(VersionOneProtocolChannelReceiver.java:213)
      	at org.jboss.as.ejb3.remote.protocol.versiontwo.VersionTwoProtocolChannelReceiver.processMessage(VersionTwoProtocolChannelReceiver.java:76)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.handleMessage(VersionOneProtocolChannelReceiver.java:159)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:456)
      	at org.jboss.remoting3.EndpointImpl$TrackingExecutor$1.run(EndpointImpl.java:717)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      	Suppressed: javax.transaction.xa.XAException
      		at org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver.sendCommit(RemotingConnectionEJBReceiver.java:513)
      		at org.jboss.ejb.client.EJBClientManagedTransactionContext$ResourceImpl.commit(EJBClientManagedTransactionContext.java:239)
      		at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelOnePhaseCommit(XAResourceRecord.java:704)
      		at com.arjuna.ats.arjuna.coordinator.BasicAction.onePhaseCommit(BasicAction.java:2366)
      		at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1495)
      		at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
      		at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
      		at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)
      		... 48 more
      	Caused by: java.io.IOException: Channel Channel ID da0eb13a (outbound) of Remoting connection 5caac271 to localhost/127.0.0.1:8280 has been closed
      		at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:123)
      		at org.jboss.ejb.client.remoting.ChannelAssociation$1.handleClose(ChannelAssociation.java:115)
      		at org.jboss.remoting3.spi.SpiUtils.safeHandleClose(SpiUtils.java:54)
      		at org.jboss.remoting3.spi.AbstractHandleableCloseable$CloseHandlerTask.run(AbstractHandleableCloseable.java:514)
      		at org.jboss.remoting3.spi.AbstractHandleableCloseable.runCloseTask(AbstractHandleableCloseable.java:419)
      		at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeComplete(AbstractHandleableCloseable.java:290)
      		at org.jboss.remoting3.remote.RemoteConnectionChannel.closeReadsAndWrites(RemoteConnectionChannel.java:274)
      		at org.jboss.remoting3.remote.RemoteConnectionChannel.closeAction(RemoteConnectionChannel.java:534)
      		at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeAsync(AbstractHandleableCloseable.java:372)
      		at org.jboss.remoting3.remote.RemoteConnectionHandler.closeAllChannels(RemoteConnectionHandler.java:437)
      		at org.jboss.remoting3.remote.RemoteConnectionHandler.receiveCloseRequest(RemoteConnectionHandler.java:217)
      		at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:111)
      		at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:46)
      		at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      		at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:199)
      		at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:113)
      		at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      		at org.xnio.ChannelListeners$DelegatingChannelListener.handleEvent(ChannelListeners.java:1092)
      		at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      		at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
      		at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:88)
      		at org.xnio.nio.WorkerThread.run(WorkerThread.java:559)
      

      This one happens when all nodes are running. It also happens in a scenario where failure isn't graceful shutdown but JVM process kill, so maybe it isn't really related to graceful shutdown.

      javax.ejb.EJBException: java.lang.IllegalStateException: EJBCLIENT000032: Cannot retry a request which hasn't previously been completed
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:187)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:277)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:79)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:53)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
      	at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
      	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.invokeMethod(MethodInvocationMessageHandler.java:327)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.access$100(MethodInvocationMessageHandler.java:67)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler$1.run(MethodInvocationMessageHandler.java:200)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.processMessage(MethodInvocationMessageHandler.java:262)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.processMessage(VersionOneProtocolChannelReceiver.java:213)
      	at org.jboss.as.ejb3.remote.protocol.versiontwo.VersionTwoProtocolChannelReceiver.processMessage(VersionTwoProtocolChannelReceiver.java:76)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.handleMessage(VersionOneProtocolChannelReceiver.java:159)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:456)
      	at org.jboss.remoting3.EndpointImpl$TrackingExecutor$1.run(EndpointImpl.java:717)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      	at ...asynchronous invocation...(Unknown Source)
      	at org.jboss.ejb.client.remoting.InvocationExceptionResponseHandler$MethodInvocationExceptionResultProducer.getResult(InvocationExceptionResponseHandler.java:96)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:276)
      	at org.jboss.ejb.client.EJBObjectInterceptor.handleInvocationResult(EJBObjectInterceptor.java:64)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290)
      	at org.jboss.ejb.client.EJBHomeInterceptor.handleInvocationResult(EJBHomeInterceptor.java:88)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290)
      	at org.jboss.ejb.client.TransactionInterceptor.handleInvocationResult(TransactionInterceptor.java:46)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290)
      	at org.jboss.ejb.client.ReceiverInterceptor.handleInvocationResult(ReceiverInterceptor.java:142)
      	at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:265)
      	at org.jboss.ejb.client.EJBClientInvocationContext.awaitResponse(EJBClientInvocationContext.java:453)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:204)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)
      	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
      	at com.sun.proxy.$Proxy26.test(Unknown Source)
      	at org.jboss.qa.eap.clustering.[redacted].tools.ClientThread.doCall(ClientThread.java:75)
      	at org.jboss.qa.eap.clustering.[redacted].tools.ClientThread.run(ClientThread.java:56)
      Caused by: java.lang.IllegalStateException: EJBCLIENT000032: Cannot retry a request which hasn't previously been completed
      	at org.jboss.ejb.client.EJBClientInvocationContext.retryRequest(EJBClientInvocationContext.java:203)
      	at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:258)
      	at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:278)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:200)
      	at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)
      	at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
      	at com.sun.proxy.$Proxy63.test(Unknown Source)
      	at org.jboss.qa.eap.clustering.[redacted].ForwardingBeanImpl.test(ForwardingBeanImpl.java:30)
      	at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
      	at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:82)
      	at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)
      	at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
      	at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
      	at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
      	at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327)
      	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.remote.EJBRemoteTransactionPropagatingInterceptor.processInvocation(EJBRemoteTransactionPropagatingInterceptor.java:79)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.deployment.processors.EjbSuspendInterceptor.processInvocation(EjbSuspendInterceptor.java:53)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
      	at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)
      	at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)
      	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
      	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
      	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.invokeMethod(MethodInvocationMessageHandler.java:327)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.access$100(MethodInvocationMessageHandler.java:67)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler$1.run(MethodInvocationMessageHandler.java:200)
      	at org.jboss.as.ejb3.remote.protocol.versionone.MethodInvocationMessageHandler.processMessage(MethodInvocationMessageHandler.java:262)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.processMessage(VersionOneProtocolChannelReceiver.java:213)
      	at org.jboss.as.ejb3.remote.protocol.versiontwo.VersionTwoProtocolChannelReceiver.processMessage(VersionTwoProtocolChannelReceiver.java:76)
      	at org.jboss.as.ejb3.remote.protocol.versionone.VersionOneProtocolChannelReceiver.handleMessage(VersionOneProtocolChannelReceiver.java:159)
      	at org.jboss.remoting3.remote.RemoteConnectionChannel$5.run(RemoteConnectionChannel.java:456)
      	at org.jboss.remoting3.EndpointImpl$TrackingExecutor$1.run(EndpointImpl.java:717)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      Links to test code and Jenkins job are in a private comment below.

            rachmato@redhat.com Richard Achmatowicz
            lthon@redhat.com Ladislav Thon
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: