Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-4182

Memory Leak When Prometheus Metrics Plugin is Enabled

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • AMQ 7.4.3.GA
    • broker-core, management
    • None
    • False
    • False
    • Undefined
    • Hide

      Disable the prometheus plugin

      Show
      Disable the prometheus plugin
    • Hide

      1. Create a vanilla AMQ 7.4.x
      2. Configure the prometheus metrics plugin, according to the documentation
      3. Run the broker
      4. Create several hundred non-durable subscriptions to different topics
      5. Publish 1 message to each topic
      6. Wait until consumers are idle and close all consumers
      7. Take a heap dump of the broker
      8. Repeat above with new topic names and growth in the number of QueueImpl objects is observed

      Show
      1. Create a vanilla AMQ 7.4.x 2. Configure the prometheus metrics plugin, according to the documentation 3. Run the broker 4. Create several hundred non-durable subscriptions to different topics 5. Publish 1 message to each topic 6. Wait until consumers are idle and close all consumers 7. Take a heap dump of the broker 8. Repeat above with new topic names and growth in the number of QueueImpl objects is observed

      When the prometheus metrics plugin is installed and enabled, creating and destroying non-durable topic subscribers results in a leak of org.apache.activemq.artemis.core.server.impl.QueueImpl objects referenced by the MetricsManager, even though the addresses and queues no longer appear in the JMX tree for the broker:

      4,202 instances of "org.apache.activemq.artemis.core.server.impl.QueueImpl", loaded by "java.net.URLClassLoader @ 0x80100000" occupy 33,971,848 (28.24%) bytes. These instances are referenced from one instance of "org.apache.activemq.artemis.core.server.metrics.MetricsManager", loaded by "java.net.URLClassLoader @ 0x80100000"

      One instance of "io.micrometer.prometheus.PrometheusMeterRegistry" loaded by "java.net.URLClassLoader @ 0x80100000" occupies 31,535,000 (26.22%) bytes. The memory is accumulated in one instance of "io.micrometer.prometheus.PrometheusMeterRegistry" loaded by "java.net.URLClassLoader @ 0x80100000".

      This issue does not seem to occur in later versions of the broker (7.5+) or with the plugin commented out.

            rhn-support-jbertram Justin Bertram
            rhn-support-dhawkins Duane Hawkins
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: