-
Bug
-
Resolution: Done
-
Critical
-
6.0.0.Final
-
None
When running CapeDwarf log tests on Infinispan 6.0.0 & Wildfly, we're seeing lots of retained instances of LocalTxInvocationContext.
So far I've seen the instances that are not removed are created at the following two points:
java.lang.Exception: Creation stack at org.infinispan.context.impl.LocalTxInvocationContext.<init>(LocalTxInvocationContext.java:42) at org.infinispan.context.TransactionalInvocationContextContainer.createInvocationContext(TransactionalInvocationContextContainer.java:100) at org.infinispan.CacheImpl.getInvocationContext(CacheImpl.java:615) at org.infinispan.CacheImpl.getInvocationContextWithImplicitTransaction(CacheImpl.java:599) at org.infinispan.CacheImpl.getInvocationContextWithImplicitTransactionForAsyncOps(CacheImpl.java:569) at org.infinispan.CacheImpl.putAsync(CacheImpl.java:996) at org.infinispan.DecoratedCache.putAsync(DecoratedCache.java:238) at org.infinispan.AbstractDelegatingCache.putAsync(AbstractDelegatingCache.java:131) at org.jboss.capedwarf.log.CapedwarfLogService$AsyncLogWriter.put(CapedwarfLogService.java:378) at org.jboss.capedwarf.log.CapedwarfLogService.requestFinished(CapedwarfLogService.java:348) at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.jboss.capedwarf.aspects.proxy.AspectContext.proceed(AspectContext.java:47) at org.jboss.capedwarf.aspects.DisableSocketsAspect.invoke(DisableSocketsAspect.java:43) at org.jboss.capedwarf.aspects.proxy.AspectWrapper.invoke(AspectWrapper.java:50) at org.jboss.capedwarf.aspects.proxy.AspectContext.proceed(AspectContext.java:52) at org.jboss.capedwarf.aspects.GlobalTimeLimitAspect.invoke(GlobalTimeLimitAspect.java:44) at org.jboss.capedwarf.aspects.proxy.AspectWrapper.invoke(AspectWrapper.java:50) at org.jboss.capedwarf.aspects.proxy.AspectContext.proceed(AspectContext.java:52) at org.jboss.capedwarf.aspects.proxy.AspectHandler.invoke(AspectHandler.java:60) at org.jboss.capedwarf.log.ExposedLogService_$$_jvst183_0.requestFinished(ExposedLogService_$$_jvst183_0.java) at org.jboss.capedwarf.appidentity.GAEListener.requestDestroyed(GAEListener.java:125) at io.undertow.servlet.core.ApplicationListeners.requestDestroyed(ApplicationListeners.java:225) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:226) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:72) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:145) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:139) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:638) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)
and
java.lang.Exception: Creation stack at org.infinispan.context.impl.LocalTxInvocationContext.<init>(LocalTxInvocationContext.java:42) at org.infinispan.context.TransactionalInvocationContextContainer.createTxInvocationContext(TransactionalInvocationContextContainer.java:110) at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:145) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:532) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:101) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1170) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306) at org.infinispan.CacheImpl.putInternal(CacheImpl.java:869) at org.infinispan.CacheImpl.put(CacheImpl.java:861) at org.infinispan.DecoratedCache.put(DecoratedCache.java:401) at org.infinispan.AbstractDelegatingCache.put(AbstractDelegatingCache.java:276) at org.jboss.capedwarf.log.CapedwarfLogService$SyncLogWriter.put(CapedwarfLogService.java:390) at org.jboss.capedwarf.log.CapedwarfLogService.requestFinished(CapedwarfLogService.java:348) at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.jboss.capedwarf.aspects.proxy.AspectContext.proceed(AspectContext.java:47) at org.jboss.capedwarf.aspects.DisableSocketsAspect.invoke(DisableSocketsAspect.java:43) at org.jboss.capedwarf.aspects.proxy.AspectWrapper.invoke(AspectWrapper.java:50) at org.jboss.capedwarf.aspects.proxy.AspectContext.proceed(AspectContext.java:52) at org.jboss.capedwarf.aspects.GlobalTimeLimitAspect.invoke(GlobalTimeLimitAspect.java:44) at org.jboss.capedwarf.aspects.proxy.AspectWrapper.invoke(AspectWrapper.java:50) at org.jboss.capedwarf.aspects.proxy.AspectContext.proceed(AspectContext.java:52) at org.jboss.capedwarf.aspects.proxy.AspectHandler.invoke(AspectHandler.java:60) at org.jboss.capedwarf.log.ExposedLogService_$$_jvst8b1_0.requestFinished(ExposedLogService_$$_jvst8b1_0.java) at org.jboss.capedwarf.appidentity.GAEListener.requestDestroyed(GAEListener.java:125) at io.undertow.servlet.core.ApplicationListeners.requestDestroyed(ApplicationListeners.java:225) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:226) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:72) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:145) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:139) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:638) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)
- relates to
-
ISPN-1764 Remove threadLocal in org.infinispan.context.AbstractInvocationContextContainer
-
- Closed
-
[ISPN-3777] ThreadLocal in AbstractInvocationContextContainer is leaking instances of LocalTxInvocationContext
If the operation is asynchronous, then the thread local is definitely not needed in the caller thread... let's try to remove it completely and see what happens.
Afaics, the first mention of removing TL was in 4.1.
Now we're at 6, and yet no TL usage is removed.
This marko.luksa@gmail.com's attempt is the first one dealing with it,
and it finally makes our testsuite pass fully.
We should get rid of this ThreadLocal: there are plenty of issues open to remove them also because of performance so that should be the cleanest solution.
Marko Luksa (Inactive)
added a comment - Well, for starters, if there's an exception before the interceptor chain is invoked, the ThreadLocal will already be set and the interceptor obviously will not be called.
Also, with async ops, the ThreadLocal is set in the calling thread, while the command and interceptors are invoked inside the async thread. (Actually, the threadlocal is never even set in the async thread. Apparently it is never used?)
Marko Luksa (Inactive)
added a comment - Well, for starters, if there's an exception before the interceptor chain is invoked, the ThreadLocal will already be set and the interceptor obviously will not be called.
Also, with async ops, the ThreadLocal is set in the calling thread, while the command and interceptors are invoked inside the async thread. (Actually, the threadlocal is never even set in the async thread. Apparently it is never used?) marko.luksa@gmail.com Why isn't it enough to clean up the ThreadLocal in an interceptor?
Marko Luksa (Inactive)
added a comment - Pull request is at https://github.com/infinispan/infinispan/pull/2263
The ThreadLocal cleanup is no longer performed by the interceptor. It is now performed inside finally.
Marko Luksa (Inactive)
added a comment - Pull request is at https://github.com/infinispan/infinispan/pull/2263
The ThreadLocal cleanup is no longer performed by the interceptor. It is now performed inside finally . Why don't you simply apply
ICC:: void clearThreadLocal();
after every
LocalTxInvocationContext ctx = icc.createTxInvocationContext();
usage.
But instead "hide" this in some Interceptor.
(where I usually see a few lines of code always being invoked before Invoker::invoke,
hence no wonder this kind of stuff starts to leak)
only one place where the thread local is set/removed.
the call classloader (either from AdvancedCache.with(ClassLoader) or from
the thread's context classloader) is the same as the configured
classloader (by default, the context classloader of the thread that
created the GlobalConfiguration).
configuration to the current thread's context classloader (but if they
are different, the context classloader was already set in the
InvocationContext and returned by EmbeddedContextClassResolver).