Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-4139

org.vertx.java.core.impl.EventLoopContext closeHooks retention

XMLWordPrintable

    • % %
    • Hide

      Duplicated in Fuse 6.2.0 P1 -Pre-patched (build 143) (Working on a duplicator for 6.2.0 P2)

      Steps to duplicate:

      1. Start Fuse and create a root/parent fabric container (mine was named "master": 

      fabric:create --wait-for-provisioning

      2. Deploy a simple AMQ/JMS route to the root container. The blueprint I used looked like:

      <?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
           xmlns:camel="http://camel.apache.org/schema/blueprint"
           xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf"	
           xsi:schemaLocation="
       http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
       http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.0.0.xsd
       http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd"
           default-activation="eager">

    <camelContext id="camel" trace="false"
                  autoStartup="true" xmlns="http://camel.apache.org/schema/blueprint"
                  useMDCLogging="true">
        
        <route id="jms-route">
            <from uri="amq://loadtestin?exchangePattern=InOnly" />
            <log loggingLevel="INFO" message="Message: ${in.body}" />
            <log loggingLevel="INFO" message="Added 2000ms delay to route." />
            <to uri="amq://loadtestout?exchangePattern=InOnly" />
        </route>    
				
    </camelContext>

</blueprint>

      3. Create a child container (mine was named "slave"):

      fabric:container-create-child master slave 1

      4. Add the gateway-mq and gateway-http profiles to the child container (Note: the http gateway may be immaterial for the test, but it was part of the tested setup).

      5. With an appropriate client (I used soapUI and hermesJMS), push messages through the gateway to the loadtestin topic and consume from the loadtestout topic. I used Simple Test Suite / Load Test 1 with settings of strategy:simple, threads:75, test delay:500 and observed a steady climb in cpu activity on the child container PID until GC Overhead Limit was encountered ~2.5 hours into the test.

      SoapUI and HermesJMS configurations are attached, as is an updated hermes-activemq library for activemq 5.9+ obtained here: (https://github.com/paoloantinori/HermesJMS/blob/fix_activemq58/activemq_fix_dist/hermes-activemq.jar). The slf4j jars in the <hermesJMS home>/lib may also need updating.

      Show
      Duplicated in Fuse 6.2.0 P1 -Pre-patched (build 143) (Working on a duplicator for 6.2.0 P2) Steps to duplicate: 1. Start Fuse and create a root/parent fabric container (mine was named "master": fabric:create --wait- for -provisioning 2. Deploy a simple AMQ/JMS route to the root container. The blueprint I used looked like: <?xml version= "1.0" encoding= "UTF-8" ?> <blueprint xmlns= "http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:camel = "http://camel.apache.org/schema/blueprint" xmlns:cxf = "http://camel.apache.org/schema/blueprint/cxf" xsi:schemaLocation=" http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.0.0.xsd http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd" default-activation= "eager" > <camelContext id= "camel" trace= "false" autoStartup= "true" xmlns= "http://camel.apache.org/schema/blueprint" useMDCLogging= "true" > <route id= "jms-route" > <from uri= "amq://loadtestin?exchangePattern=InOnly" /> <log loggingLevel= "INFO" message= "Message: ${in.body}" /> <log loggingLevel= "INFO" message= "Added 2000ms delay to route." /> <to uri= "amq://loadtestout?exchangePattern=InOnly" /> </route> </camelContext> </blueprint> 3. Create a child container (mine was named "slave"): fabric:container-create-child master slave 1 4. Add the gateway-mq and gateway-http profiles to the child container (Note: the http gateway may be immaterial for the test, but it was part of the tested setup). 5. With an appropriate client (I used soapUI and hermesJMS), push messages through the gateway to the loadtestin topic and consume from the loadtestout topic. I used Simple Test Suite / Load Test 1 with settings of strategy:simple, threads:75, test delay:500 and observed a steady climb in cpu activity on the child container PID until GC Overhead Limit was encountered ~2.5 hours into the test. SoapUI and HermesJMS configurations are attached, as is an updated hermes-activemq library for activemq 5.9+ obtained here: ( https://github.com/paoloantinori/HermesJMS/blob/fix_activemq58/activemq_fix_dist/hermes-activemq.jar ). The slf4j jars in the <hermesJMS home>/lib may also need updating.

      A Fabric child container with gateway-http and gateway-mq profiles installed runs out of memory (GC overhead). Heap dump shows heap retention at org.vertx.java.core.impl.EventLoopContext's closeHooks:

      Class Name                                                 | Shallow Heap | Retained Heap
------------------------------------------------------------------------------------------
org.vertx.java.core.impl.EventLoopContext @ 0xe155e5b0     |         48 B |     408.22 MB
|- closeHooks java.util.HashSet @ 0xe155e5e0               |         16 B |     408.22 MB
|  |- map java.util.HashMap @ 0xe155e5f0                   |         48 B |     408.22 MB
|  |  |- table java.util.HashMap$Node[1048576] @ 0xfbc10dd0|      4.00 MB |     408.22 MB
|  |  |  |- [905684] java.util.HashMap$Node @ 0xe30963a0   |         32 B |       5.74 KB
|  |  |  |- [444796] java.util.HashMap$Node @ 0xe1a7dd18   |         32 B |       4.92 KB
|  |  |  |- [994687] java.util.HashMap$Node @ 0xe1de9570   |         32 B |       4.92 KB
|  |  |  |- [847173] java.util.HashMap$Node @ 0xe207fe98   |         32 B |       4.92 KB
|  |  |  |- [283725] java.util.HashMap$Node @ 0xe21b7c50   |         32 B |       4.92 KB
|  |  |  |- [710891] java.util.HashMap$Node @ 0xe2339310   |         32 B |       4.92 KB
|  |  |  |- [889097] java.util.HashMap$Node @ 0xe24d33c0   |         32 B |       4.92 KB
|  |  |  |- [920100] java.util.HashMap$Node @ 0xe292efb0   |         32 B |       4.92 KB
|  |  |  |- [419150] java.util.HashMap$Node @ 0xe3bd6c30   |         32 B |       4.92 KB
|  |  |  |- [20246] java.util.HashMap$Node @ 0xe4893680    |         32 B |       4.92 KB
|  |  |  |- [480642] java.util.HashMap$Node @ 0xe6b9c920   |         32 B |       4.92 KB
|  |  |  |- [694888] java.util.HashMap$Node @ 0xe779fe00   |         32 B |       4.92 KB
|  |  |  |- [126585] java.util.HashMap$Node @ 0xe7b06798   |         32 B |       4.92 KB
|  |  |  |- [72051] java.util.HashMap$Node @ 0xe810c170    |         32 B |       4.92 KB
|  |  |  |- [159256] java.util.HashMap$Node @ 0xe87e4a30   |         32 B |       4.92 KB
|  |  |  |- [923394] java.util.HashMap$Node @ 0xed5726c8   |         32 B |       4.92 KB
|  |  |  |- [308446] java.util.HashMap$Node @ 0xf1272f60   |         32 B |       4.92 KB
|  |  |  |- [377290] java.util.HashMap$Node @ 0xe17e08d0   |         32 B |       4.10 KB
|  |  |  |- [80413] java.util.HashMap$Node @ 0xe17f6b30    |         32 B |       4.10 KB
|  |  |  |- [857848] java.util.HashMap$Node @ 0xe18306c0   |         32 B |       4.10 KB
|  |  |  |- [1042560] java.util.HashMap$Node @ 0xe1845c60  |         32 B |       4.10 KB
|  |  |  |- [210838] java.util.HashMap$Node @ 0xe19168f8   |         32 B |       4.10 KB
|  |  |  |- [839023] java.util.HashMap$Node @ 0xe19f3710   |         32 B |       4.10 KB
|  |  |  |- [400432] java.util.HashMap$Node @ 0xe1a6f758   |         32 B |       4.10 KB
|  |  |  |- [888719] java.util.HashMap$Node @ 0xe1a842e8   |         32 B |       4.10 KB
|  |  |  '- Total: 25 of 400,482 entries; 400,457 more     |              |              
------------------------------------------------------------------------------------------

        1. gateway-heap.png
          gateway-heap.png
          98 kB
        2. hermes-activemq.jar
          6 kB
        3. hermes-config.xml
          15 kB
        4. Sample-SOAP-Project-soapui-project.xml
          141 kB
        5. soapui-settings.xml
          13 kB
        6. http-amq-camel-test.tar.gz
          3 kB

            hchirino Hiram Chirino
            rhn-support-iweiss Ingo Weiss
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: