Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-695

7.0.2 patch install causes container to hang

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Do
    • Major
    • M1
    • fuse-esb-7.0.2
    • None
    • None
    • % %

    Description

      During patch testing for 7.0.2 I was able see the container occasionally hang. From the thread dumps I see the following:

      "pool-org.apache.karaf.deployer.features-2.2.5.fuse-70-097-thread-3" prio=5 tid=7fa07dacd800 nid=0x10ecf6000 waiting for monitor entry [10ecf4000]
         java.lang.Thread.State: BLOCKED (on object monitor)
      	at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:212)
      	- waiting to lock <7faf399d8> (a java.lang.Object)
      	at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:147)
      	at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:641)
      	at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:347)
      	at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:240)
      	- locked <7fafd7518> (a java.util.concurrent.atomic.AtomicBoolean)
      	at org.apache.aries.blueprint.container.BlueprintExtender.checkBundle(BlueprintExtender.java:325)
      	at org.apache.aries.blueprint.container.BlueprintExtender.bundleChanged(BlueprintExtender.java:244)
      	at org.apache.aries.blueprint.container.BlueprintExtender$BlueprintBundleTrackerCustomizer.modifiedBundle(BlueprintExtender.java:471)
      	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:495)
      	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:1)
      	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:238)
      	at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:457)
      	at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:870)
      	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:791)
      	at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:515)
      	at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4493)
      	at org.apache.felix.framework.Felix$4.run(Felix.java:2019)
      	at org.apache.felix.framework.Felix$5.run(Felix.java:2061)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      	at java.lang.Thread.run(Thread.java:680)
      

      Which is waiting to lock an object held by:

      "pool-org.apache.karaf.deployer.features-2.2.5.fuse-70-097-thread-4" prio=5 tid=7fa07938e000 nid=0x115bbc000 in Object.wait() [115bba000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <7e0207640> (a [Ljava.lang.Object;)
      	at java.lang.Object.wait(Object.java:485)
      	at org.apache.felix.framework.Felix.acquireGlobalLock(Felix.java:5271)
      	- locked <7e0207640> (a [Ljava.lang.Object;)
      	at org.apache.felix.framework.Felix.resolveBundles(Felix.java:3983)
      	at org.apache.felix.framework.Felix.findBundleEntries(Felix.java:1713)
      	at org.apache.felix.framework.BundleImpl.findEntries(BundleImpl.java:306)
      	at org.apache.felix.framework.Felix.findEntries(Felix.java:89)
      	at org.apache.karaf.deployer.features.FeatureDeploymentListener.bundleChanged(FeatureDeploymentListener.java:166)
      	at org.apache.karaf.deployer.features.FeatureDeploymentListener.init(FeatureDeploymentListener.java:91)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.apache.aries.blueprint.utils.ReflectionUtils.invoke(ReflectionUtils.java:284)
      	at org.apache.aries.blueprint.container.BeanRecipe.invoke(BeanRecipe.java:839)
      	at org.apache.aries.blueprint.container.BeanRecipe.runBeanProcInit(BeanRecipe.java:639)
      	at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:727)
      	at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:64)
      	- locked <7faf399d8> (a java.lang.Object)
      	at org.apache.aries.blueprint.di.RefRecipe.internalCreate(RefRecipe.java:60)
      	at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:64)
      	- locked <7faf399d8> (a java.lang.Object)
      	at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:219)
      	- locked <7faf399d8> (a java.lang.Object)
      	at org.apache.aries.blueprint.container.BlueprintRepository.createInstance(BlueprintRepository.java:198)
      	at org.apache.aries.blueprint.container.BlueprintRepository.create(BlueprintRepository.java:137)
      	at org.apache.aries.blueprint.container.ServiceRecipe.createRecipe(ServiceRecipe.java:370)
      	at org.apache.aries.blueprint.container.ServiceRecipe.createService(ServiceRecipe.java:278)
      	at org.apache.aries.blueprint.container.ServiceRecipe.internalGetService(ServiceRecipe.java:248)
      	- locked <7faf399d8> (a java.lang.Object)
      	at org.apache.aries.blueprint.container.ServiceRecipe.internalGetService(ServiceRecipe.java:229)
      	at org.apache.aries.blueprint.container.ServiceRecipe.getClasses(ServiceRecipe.java:340)
      	at org.apache.aries.blueprint.container.ServiceRecipe.register(ServiceRecipe.java:178)
      	at org.apache.aries.blueprint.container.BlueprintContainerImpl.notifySatisfaction(BlueprintContainerImpl.java:613)
      	- locked <7faf39ac8> (a org.apache.aries.blueprint.container.ServiceRecipe)
      	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe.setSatisfied(AbstractServiceReferenceRecipe.java:304)
      	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe.serviceAdded(AbstractServiceReferenceRecipe.java:261)
      	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe.access$000(AbstractServiceReferenceRecipe.java:64)
      	at org.apache.aries.blueprint.container.AbstractServiceReferenceRecipe$1.run(AbstractServiceReferenceRecipe.java:239)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      	at java.lang.Thread.run(Thread.java:680)
      

      On the console the following output occurs:

      FuseESB:karaf@root> patch:install fuse-esb-7.0.2.fuse-097-p5
      Bundles to update:
          org.apache.karaf.admin.core/2.2.5.fuse-70-097 with mvn:org.apache.karaf.admin/org.apache.karaf.admin.core/2.2.5.fuse-70-102
          org.apache.karaf.shell.ssh/2.2.5.fuse-70-097 with mvn:org.apache.karaf.shell/org.apache.karaf.shell.ssh/2.2.5.fuse-70-102
          org.apache.karaf.shell.console/2.2.5.fuse-70-097 with mvn:org.apache.karaf.shell/org.apache.karaf.shell.console/2.2.5.fuse-70-102
          org.apache.camel.camel-jetty/2.9.0.fuse-70-097 with mvn:org.apache.camel/camel-jetty/2.9.0.fuse-70-107
          org.apache.camel.camel-http/2.9.0.fuse-70-097 with mvn:org.apache.camel/camel-http/2.9.0.fuse-70-107
          org.apache.mina.core/2.0.4 with mvn:org.apache.mina/mina-core/2.0.4.fuse-70-102
          org.apache.karaf.admin.command/2.2.5.fuse-70-097 with mvn:org.apache.karaf.admin/org.apache.karaf.admin.command/2.2.5.fuse-70-102
      Installation will begin.  The connection may be lost or the console restarted.
      
      
      [Blueprint Extender: 1] ERROR org.apache.aries.blueprint.container.BlueprintContainerImpl - Unable to start blueprint container for bundle org.apache.karaf.shell.shell due to unresolved dependencies [(objectClass=org.apache.felix.service.command.CommandProcessor)]
      java.util.concurrent.TimeoutException
      	at org.apache.aries.blueprint.container.BlueprintContainerImpl$1.run(BlueprintContainerImpl.java:303)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      	at java.lang.Thread.run(Thread.java:680)
      [Blueprint Extender: 1] ERROR org.apache.aries.blueprint.container.BlueprintContainerImpl - Unable to start blueprint container for bundle org.apache.karaf.admin.management due to unresolved dependencies [(objectClass=org.apache.karaf.admin.AdminService), (objectClass=org.apache.felix.service.threadio.ThreadIO)]
      java.util.concurrent.TimeoutException
      	at org.apache.aries.blueprint.container.BlueprintContainerImpl$1.run(BlueprintContainerImpl.java:303)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      	at java.lang.Thread.run(Thread.java:680)
      

      Attachments

        1. dump.txt
          35 kB
        2. dump2.txt
          45 kB
        3. dump3.txt
          44 kB

        Activity

          People

            janstey@redhat.com Jonathan Anstey
            rhn-support-jsherman Jason Sherman
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: