Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-2056

exception in stopping a component resulted in failing to stop cache manager

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: 5.1.4.FINAL
    • Fix Version/s: 7.0.0.Beta2
    • Component/s: Core
    • Labels:
      None
    • Environment:

      ISPN 5.1.4.Final
      2 node cluster with jgroup-tcp.xml and replication mode

      Description

      If cacheManager.stop() got any exception, the cache manger will not be able stopped by retrying stop().

      This will resulted in following new cache manager instance with same configure fail to start with following exception:

      org.infinispan.CacheException: Unable to invoke method public void
      org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on
      object of type JGroupsTransport
              at
      org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:238)
              at
      org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:885)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:639)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:226)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:152)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:167)
              at
      org.infinispan.factories.InterceptorChainFactory.register(InterceptorChainFactory.java:73)
              at
      org.infinispan.factories.InterceptorChainFactory.createInterceptor(InterceptorChainFactory.java:62)
              at
      org.infinispan.factories.InterceptorChainFactory.buildInterceptorChain(InterceptorChainFactory.java:95)
              at
      org.infinispan.factories.InterceptorChainFactory.construct(InterceptorChainFactory.java:272)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:286)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.AbstractComponentRegistry.getOrCreateComponent(AbstractComponentRegistry.java:291)
              at
      org.infinispan.factories.AbstractComponentRegistry.invokeInjectionMethod(AbstractComponentRegistry.java:246)
              at
      org.infinispan.factories.AbstractComponentRegistry.access$000(AbstractComponentRegistry.java:83)
              at
      org.infinispan.factories.AbstractComponentRegistry$Component.injectDependencies(AbstractComponentRegistry.java:792)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponentInternal(AbstractComponentRegistry.java:220)
              at
      org.infinispan.factories.ComponentRegistry.registerComponentInternal(ComponentRegistry.java:154)
              at
      org.infinispan.factories.AbstractComponentRegistry.registerComponent(AbstractComponentRegistry.java:175)
              at
      org.infinispan.factories.InternalCacheFactory.bootstrap(InternalCacheFactory.java:102)
              at
      org.infinispan.factories.InternalCacheFactory.createAndWire(InternalCacheFactory.java:78)
              at
      org.infinispan.factories.InternalCacheFactory.createCache(InternalCacheFactory.java:62)
              at
      org.infinispan.manager.DefaultCacheManager.wireCache(DefaultCacheManager.java:692)
              at
      org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)
              at
      org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:549)
              at
      .....StatusUpdater.poll.....
      
      ....
      
      Caused by: org.infinispan.CacheException: Unable to start JGroups Channel
              at
      org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:186)
              at
      org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:175)
              at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source)
              at
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:601)
              at
      org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:236)
              ... 96 more
      
      Caused by: java.lang.IllegalStateException: cluster 'HA_Customer_9.3' is
      already connected to singleton transport: [dummy-1336085803144,
      dummy-1336085899401, dummy-1336085785840, dummy-1336085828476,
      dummy-1336085808197, HA_Customer_9.3, dummy-1336085874063,
      dummy-1336085868964, dummy-1336085785779, dummy-1336085818352,
      dummy-1336085792988, dummy-1336085879152, dummy-1336085823409,
      dummy-1336085785693, dummy-1336085848698, dummy-1336085785734,
      dummy-1336085813295, dummy-1336085889257, dummy-1336085853769,
      dummy-1336085894308, dummy-1336085843643, dummy-1336085833534,
      dummy-1336085787888, dummy-1336085798048, dummy-1336085785653,
      dummy-1336085884206, dummy-1336085863914, dummy-1336085785603,
      dummy-1336085838588, dummy-1336085858821]
              at org.jgroups.stack.ProtocolStack.startStack(ProtocolStack.java:909)
              at org.jgroups.JChannel.startStack(JChannel.java:841)
              at org.jgroups.JChannel.connect(JChannel.java:277)
              at org.jgroups.JChannel.connect(JChannel.java:261)
              at
      org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:184)
      

      This specific case is caused by cache manager can not terminate the first cache manager instance becaue of SyncFailedException: sync failed as shown:

       2012-05-17/10:11:32.539/MDT [Thread-23] DEBUG com.symplified.service.shared.platform.cluster.InfinispanClusterCacheStoreServiceImpl[183] - exception caught in stop existing cache manager
      org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.stop() on object of type CacheLoaderManagerImpl
              at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:238)
              at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:885)
              at org.infinispan.factories.AbstractComponentRegistry.internalStop(AbstractComponentRegistry.java:674)
              at org.infinispan.factories.AbstractComponentRegistry.stop(AbstractComponentRegistry.java:553)
              at org.infinispan.factories.ComponentRegistry.stop(ComponentRegistry.java:199)
              at org.infinispan.CacheImpl.stop(CacheImpl.java:539)
              at org.infinispan.CacheImpl.stop(CacheImpl.java:531)
              at org.infinispan.manager.DefaultCacheManager.stop(DefaultCacheManager.java:742)
      
      .....
      
      Caused by: org.infinispan.CacheException: java.io.SyncFailedException: sync failed
              at org.infinispan.loaders.jdbm.JdbmCacheStore.stop(JdbmCacheStore.java:246)
              at org.infinispan.loaders.decorators.AbstractDelegatingStore.stop(AbstractDelegatingStore.java:155)
              at org.infinispan.loaders.decorators.AsyncStore.stop(AsyncStore.java:237)
              at org.infinispan.loaders.CacheLoaderManagerImpl.stop(CacheLoaderManagerImpl.java:221)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:601)
              at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:236)
              ... 14 more
      
      Caused by: java.io.SyncFailedException: sync failed
              at java.io.FileDescriptor.sync(Native Method)
              at jdbm.recman.TransactionManager.sync(TransactionManager.java:331)
              at jdbm.recman.TransactionManager.close(TransactionManager.java:347)
              at jdbm.recman.TransactionManager.synchronizeLogFromMemory(TransactionManager.java:156)
              at jdbm.recman.TransactionManager.shutdown(TransactionManager.java:339)
              at jdbm.recman.RecordFile.close(RecordFile.java:346)
              at jdbm.recman.BaseRecordManager.close(BaseRecordManager.java:168)
      

      I do not see there is any way to froce stop the cache manager.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                galder.zamarreno Galder ZamarreƱo
                Reporter:
                dex80526 dex chen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: