Uploaded image for project: 'AMQ Interconnect'
  1. AMQ Interconnect
  2. ENTMQIC-61

Memory pools are never returned to heap

    Details

    • Type: Bug
    • Status: New
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0
    • Fix Version/s: None
    • Component/s: Qpid Dispatch Router
    • Labels:
      None
    • Environment:

      Linux

    • Target Release:
    • Steps to Reproduce:
      Hide

      use listener/sender tool - send 6000000 messages, router link credit 2500, maxframe 4K

      Show
      use listener/sender tool - send 6000000 messages, router link credit 2500, maxframe 4K
    • Affects:
      Release Notes
    • Release Notes Text:
      Hide
      Several heavily used data objects (deliveries, messages, links, buffers, etc.) are managed by Qpid Dispatch Router in pools for efficient allocation. In AMQ Interconnect 1.1, objects in these pools are not returned to the heap at any time. This means that the memory used in large bursts of activity will not be freed, but will remain available for use thereafter.
      This might be observed as an increase in memory usage that does not decrease after a burst of activity is completed. Subsequent bursts of activity will use the same memory that was used previously.
      Methods of returning large amounts of pooled objects back to the heap are being developed and a solution will be included in a future version of AMQ Interconnect.
      Show
      Several heavily used data objects (deliveries, messages, links, buffers, etc.) are managed by Qpid Dispatch Router in pools for efficient allocation. In AMQ Interconnect 1.1, objects in these pools are not returned to the heap at any time. This means that the memory used in large bursts of activity will not be freed, but will remain available for use thereafter. This might be observed as an increase in memory usage that does not decrease after a burst of activity is completed. Subsequent bursts of activity will use the same memory that was used previously. Methods of returning large amounts of pooled objects back to the heap are being developed and a solution will be included in a future version of AMQ Interconnect.
    • Release Notes Docs Status:
      Documented as Known Issue

      Description

      After running millions of messages through the router, the message related pooled structures are never released to the heap:

      1. qdstat -m
        Types
        type size batch thread-max total in-threads rebal-in rebal-out
        =============================================================================================
        qd_bitmask_t 24 64 128 64 64 0 0
        qd_buffer_t 536 16 32 4,287,264 560 472,869 740,788
        qd_composed_field_t 64 64 128 256 256 0 0
        qd_composite_t 112 64 128 256 256 0 0
        qd_connection_t 224 64 128 256 256 0 0
        qd_deferred_call_t 32 64 128 256 256 0 0
        qd_field_iterator_t 128 64 128 704 704 6 6
        qd_hash_handle_t 16 64 128 64 64 0 0
        qd_hash_item_t 32 64 128 128 128 0 0
        qd_hash_segment_t 24 64 128 64 64 0 0
        qd_link_t 48 64 128 256 256 0 0
        qd_listener_t 32 64 128 64 64 0 0
        qd_log_entry_t 2,104 16 32 32 32 0 0
        qd_message_content_t 640 16 32 1,071,872 80 111,216 178,203
        qd_message_t 128 64 128 1,072,192 448 21,630 38,376
        qd_node_t 56 64 128 64 64 0 0
        qd_parsed_field_t 80 64 128 256 256 0 0
        qd_timer_t 56 64 128 128 128 0 0
        qd_work_item_t 24 64 128 320 256 8 9
        qdpn_connector_t 600 16 32 80 64 0 1
        qdpn_listener_t 48 64 128 64 64 0 0
        qdr_action_t 128 64 128 384 256 48,795 48,797
        qdr_address_config_t 56 64 128 64 64 0 0
        qdr_address_t 224 64 128 64 64 0 0
        qdr_connection_t 200 64 128 320 192 4 6
        qdr_connection_work_t 56 64 128 512 384 7 9
        qdr_delivery_ref_t 24 64 128 448 448 1 1
        qdr_delivery_t 96 64 128 1,072,192 448 21,628 38,374
        qdr_error_t 24 64 128 320 192 11 13
        qdr_field_t 40 64 128 704 704 6 6
        qdr_general_work_t 64 64 128 512 384 7 9
        qdr_link_ref_t 24 64 128 576 448 17,539 17,541
        qdr_link_t 264 16 32 80 64 54 55
        qdr_query_t 336 16 32 16 16 0 0
        qdr_terminus_t 64 64 128 256 192 1 2

      And

      ]# qdmanage query --type allocator | grep -i freetoheap
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,
      "totalFreeToHeap": 0,

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kgiusti Kenneth Giusti
                Tester:
                Matej Lesko
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: