ReplicationQueueImpl shuts down shared component (a scheduled executor) when stopped

This is incorrect behaviour as the shared component could still be used by other caches. The task submitted to the shared executor should simply be de-registered instead.

Original bug report

After enabling the replication queue for REPL_ASYNC cache in JBoss AS6, I get the following exception during startup of the cache:

 
2011-01-25 15:17:53,127 ERROR [org.jboss.web.tomcat.service.session.JBossCacheManager.http-attr-pass] (RMI TCP Connection(3)-127.0.0.1) Unable to start manager.: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.remoting.ReplicationQueueImpl.start() on object
        at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:174) [:4.2.0.FINAL]
        at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:889) [:4.2.0.FINAL]
        at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:687) [:4.2.0.FINAL]
        at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:589) [:4.2.0.FINAL]
        at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:150) [:4.2.0.FINAL]
        at org.infinispan.CacheDelegate.start(CacheDelegate.java:317) [:4.2.0.FINAL]
        at org.infinispan.AbstractDelegatingCache.start(AbstractDelegatingCache.java:251) [:4.2.0.FINAL]
        at org.jboss.web.tomcat.service.session.distributedcache.ispn.DistributedCacheManager.start(DistributedCacheManager.java:137) [:1.0.0.Final]
        at org.jboss.web.tomcat.service.session.JBossCacheManager.startEmbedded(JBossCacheManager.java:2697) [:6.0.1-SNAPSHOT]
        at org.jboss.web.tomcat.service.session.JBossCacheManager.start(JBossCacheManager.java:828) [:6.0.1-SNAPSHOT]
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:3837) [:]
        at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeployInternal(TomcatDeployment.java:294) [:6.0.1-SNAPSHOT]
        at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeploy(TomcatDeployment.java:146) [:6.0.1-SNAPSHOT]
        at org.jboss.web.deployers.AbstractWarDeployment.start(AbstractWarDeployment.java:477) [:6.0.1-SNAPSHOT]
        at org.jboss.web.deployers.WebModule.startModule(WebModule.java:118) [:6.0.1-SNAPSHOT]
        at org.jboss.web.deployers.WebModule.start(WebModule.java:95) [:6.0.1-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_21]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_21]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_21]
        at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_21]
        at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157) [:6.0.0.GA]
        at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96) [:6.0.0.GA]
        at org.jboss.mx.server.Invocation.invoke(Invocation.java:88) [:6.0.0.GA]
        at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:271) [:6.0.0.GA]
        at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:670) [:6.0.0.GA]
        at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:206) [:2.2.0.GA]
        at $Proxy41.start(Unknown Source)       at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:53) [:2.2.0.GA]

        at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:41) [:2.2.0.GA]
        at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:379) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:301) [:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:2044) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:1083) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.executeOrIncrementStateDirectly(AbstractController.java:1322) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1246) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1139) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:894) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:641) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:182) [:2.2.0.GA]
        at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:58) [:2.2.0.GA]
        at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62) [:2.2.0.GA]
        at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:55) [:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:179) [:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1832) [:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1550) [:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1571) [:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1491) [:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:379) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:2044) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:1083) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.executeOrIncrementStateDirectly(AbstractController.java:1322) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1246) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1139) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:939) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:654) [jboss-dependency.jar:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployersImpl.change(DeployersImpl.java:1983) [:2.2.0.GA]
        at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:1076) [:2.2.0.GA]
        at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:679) [:2.2.0.GA]
        at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:380) [:6.0.1-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_21]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_21]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_21]
        at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_21]

        at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157) [:6.0.0.GA]
        at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96) [:6.0.0.GA]
        at org.jboss.mx.server.Invocation.invoke(Invocation.java:88) [:6.0.0.GA]
        at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:271) [:6.0.0.GA]
        at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:670) [:6.0.0.GA]
        at org.jboss.system.server.jmx.MBeanServerWrapper.invoke(MBeanServerWrapper.java:138) [:6.0.1-SNAPSHOT (Build SVNTag:JBoss_6.0.1-SNAPSHOT date: 20110125)]
        at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1450) [:1.6.0_21]
        at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:90) [:1.6.0_21]
        at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1285) [:1.6.0_21]
        at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1383) [:1.6.0_21]
        at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:807) [:1.6.0_21]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_21]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_21]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_21]
        at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_21]
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322) [:1.6.0_21]
        at sun.rmi.transport.Transport$1.run(Transport.java:177) [:1.6.0_21]
        at java.security.AccessController.doPrivileged(Native Method) [:1.6.0_21]
        at sun.rmi.transport.Transport.serviceCall(Transport.java:173) [:1.6.0_21]
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553) [:1.6.0_21]
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808) [:1.6.0_21]
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667) [:1.6.0_21]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [:1.6.0_21]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [:1.6.0_21]
        at java.lang.Thread.run(Thread.java:679) [:1.6.0_21]
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_21]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_21]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_21]
        at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_21]
        at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:171) [:4.2.0.FINAL]
        ... 90 more
Caused by: java.util.concurrent.RejectedExecutionException
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1956) [:1.6.0_21]
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816) [:1.6.0_21]
        at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:299) [:1.6.0_21]
        at java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:520) [:1.6.0_21]
        at java.util.concurrent.Executors$DelegatedScheduledExecutorService.scheduleWithFixedDelay(Executors.java:699) [:1.6.0_21]
        at org.infinispan.remoting.ReplicationQueueImpl.start(ReplicationQueueImpl.java:81) [:4.2.0.FINAL]
        ... 95 more

This is odd, because tracing the ScheduledThreadPoolExecutor code, this would only happen if the executor was already shutdown.