Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-8477

AMQ 7: heap is not cleared of OpenWireConnection objects after all connections are closed -- with Windows client

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Undefined
    • None
    • AMQ 7.11.1.GA
    • openwire-protocol
    • None
    • False
    • None
    • False
    • ARTEMIS-4731
    • Moderate

    Description

      AMQ 7.11 on Linux, JDK 17. OpenWire Java client on Linux or Windows, JDK 17. With effort, the client on Windows can cause the broker to run out of heap. This behaviour does not reproduce with the client on Linux.

      The client opens thousands of connections to the broker, using the OpenWire client runtime, and then exits without cleaning up. Essentially the client code is:

      ConnectionFactory factory = factory = ActiveMQConnectionFactory (url);
      for (int i = 0; i < 10000; i++)
          factory.createConnection(user, pass);
      System.exit (0);
        

      Eventually all the sockets that the client opened on the broker get cleaned up, so `netstat` reveals no sockets except the listener – nothing in TIME_WAIT, nothing ESTABLISHED. With the client running on Linux, it may take some time (minutes) to get to this state, once the client has finished. With the client on Windows, it seemingly takes no time at all.

      Although all sockets are closed, when the client has been run on Windows, a broker heap dump shows thousands of objects of class `OpenWireConnection` and related artefacts. The heap does not reduce to its pre-test level, even if GC is forced. Further runs of the test increase the number of these objects. Eventually, the broker runs out of heap.

      This problem does not reproduce with the client on Linux.

      I note the following differences between running the client on Linux and Windows.

      1. On Linux, creating new connections to the broker is faster – tens of times faster – than with the client on Windows.
      2.  On Linux, when the client completes, thousands of sockets are left in TIME_WAIT, on both the broker and the client host. That's kind-of expected when the client does not clean up properly, but I don't see it with the Windows client. With the Windows client, as soon as it completes, all sockets appear to be closed.

      I speculate that there there is something in the sequence of operations that occur during TCP clean-up, that confuse the broker into leaving OpenWire-related artefacts still reachable, and not elligible for GC.

       

       

       

       

       

      Attachments

        Issue Links

          Activity

            People

              csuconic@redhat.com Clebert Suconic
              rhn-support-kboone Kevin Boone
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: