Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-9626

Race Condition Trying to Load profile:broker.xml after Rollup 7 is Applied

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • fuse-6.x-GA
    • fuse-6.x-GA
    • ActiveMQ, Fabric8 v1
    • None
    • % %
    • Hide
      • Create a 3-node fabric cluster with an earlier version if Fuse 6.3 (I started with Rollup 2/630254)
      • Create master/slave broker profiles in a hierarchy similar to that of the "stage" profiles in the attached export
      • Create 6 broker child containers and assign each master/slave profile to a container
      • Follow the documented instructions to upgrade the patch-management components to version ending in 630343.
      • Follow the documented instructions to upgrade the cluster to Fuse 6.3.0 / Rollup 7 / 630343.
      • Restart broker containers while tailing the logs. The referenced exception is thrown upon first startup attempt. The configuration is later resolved and the broker is able to start.
      Show
      Create a 3-node fabric cluster with an earlier version if Fuse 6.3 (I started with Rollup 2/630254) Create master/slave broker profiles in a hierarchy similar to that of the "stage" profiles in the attached export Create 6 broker child containers and assign each master/slave profile to a container Follow the documented instructions to upgrade the patch-management components to version ending in 630343. Follow the documented instructions to upgrade the cluster to Fuse 6.3.0 / Rollup 7 / 630343. Restart broker containers while tailing the logs. The referenced exception is thrown upon first startup attempt. The configuration is later resolved and the broker is able to start.

    Description

      After applying Fuse 6.3.0 Rollup 7 to an existing Fuse environment with a 3-node ensemble and master/slave broker profiles, the broker containers take a longer time to start and from the log we can see that the ActiveMQServiceFactory cannot find the configured broker.xml file on first start attempt and fails with FileNotFoundException:

      2018-10-13 16:32:38,707 | ERROR | AMQ-1-thread-1   | ActiveMQServiceFactory           | 105 - io.fabric8.mq.mq-fabric - 1.2.0.redhat-630343 | Exception on start: org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from class path resource [profile:stage-amq-broker3.xml]; nested exception is java.io.FileNotFoundException: class path resource [profile:stage-amq-broker3.xml] cannot be opened because it does not exist
org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from class path resource [profile:stage-amq-broker3.xml]; nested exception is java.io.FileNotFoundException: class path resource [profile:stage-amq-broker3.xml] cannot be opened because it does not exist
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:342)[127:org.apache.servicemix.bundles.spring-beans:3.2.18.RELEASE_1]
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:303)[127:org.apache.servicemix.bundles.spring-beans:3.2.18.RELEASE_1]
	at org.apache.xbean.spring.context.ResourceXmlApplicationContext.loadBeanDefinitions(ResourceXmlApplicationContext.java:111)[138:org.apache.xbean.spring:3.18.0]
	at org.apache.xbean.spring.context.ResourceXmlApplicationContext.loadBeanDefinitions(ResourceXmlApplicationContext.java:104)[138:org.apache.xbean.spring:3.18.0]
	at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:130)[128:org.apache.servicemix.bundles.spring-context:3.2.18.RELEASE_1]
	at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:545)[128:org.apache.servicemix.bundles.spring-context:3.2.18.RELEASE_1]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:454)[128:org.apache.servicemix.bundles.spring-context:3.2.18.RELEASE_1]
	at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)[138:org.apache.xbean.spring:3.18.0]
	at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)[138:org.apache.xbean.spring:3.18.0]
	at io.fabric8.mq.fabric.ActiveMQServiceFactory$1.<init>(ActiveMQServiceFactory.java:164)[105:io.fabric8.mq.mq-fabric:1.2.0.redhat-630343]
	at io.fabric8.mq.fabric.ActiveMQServiceFactory.createBroker(ActiveMQServiceFactory.java:164)[105:io.fabric8.mq.mq-fabric:1.2.0.redhat-630343]
	at io.fabric8.mq.fabric.ActiveMQServiceFactory$ClusteredConfiguration.doStart(ActiveMQServiceFactory.java:543)[105:io.fabric8.mq.mq-fabric:1.2.0.redhat-630343]
	at io.fabric8.mq.fabric.ActiveMQServiceFactory$ClusteredConfiguration.access$500(ActiveMQServiceFactory.java:359)[105:io.fabric8.mq.mq-fabric:1.2.0.redhat-630343]
	at io.fabric8.mq.fabric.ActiveMQServiceFactory$ClusteredConfiguration$1.run(ActiveMQServiceFactory.java:498)[105:io.fabric8.mq.mq-fabric:1.2.0.redhat-630343]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_181]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748)[:1.8.0_181]
Caused by: java.io.FileNotFoundException: class path resource [profile:stage-amq-broker3.xml] cannot be opened because it does not exist
	at org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:171)[130:org.apache.servicemix.bundles.spring-core:3.2.18.RELEASE_1]
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:329)[127:org.apache.servicemix.bundles.spring-beans:3.2.18.RELEASE_1]
	... 18 more

      On the next retry 10 seconds later, the file is found and the broker configures and boots up.

      Attachments

        1. config.tar
          460 kB
        2. etc.tar
          150 kB
        3. fuse.log
          377 kB
        4. fuse.log
          122 kB
        5. heap.hprof.gz
          19.67 MB
        6. mq-profiles.zip
          44 kB

        Issue Links

          is incorporated by

          Bug - A problem that impairs or prevents the functions of the product. ENTESB-7843 Jetty unable to find its SSL keystore in Fabric registry: FileNotFoundException: /opt/fuse/jboss-fuse-6.3/profile:fuse.jks (No such file or directory)

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. ENTESB-7259 Fabric container tries to create two broker instances

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. ENTESB-7515 AMQ broker is not started after patching to 6.3 R6

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Activity

            People

              ggrzybek Grzegorz Grzybek
              rhn-support-dhawkins Duane Hawkins
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: