Uploaded image for project: 'JBoss A-MQ'
  1. JBoss A-MQ
  2. ENTMQ-2195

Missing or Stuck Network Queue Consumers in Network of Brokers with conduitSubscriptions Disabled

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • JBoss A-MQ 6.3.x
    • JBoss A-MQ 6.3
    • broker, networks
    • None
    • +
    • Hide

      We have been able to intermittently reproduce this by starting a 3-broker NOB and bringing up and down large numbers of consumers, waiting for them to balance and we see that the consumer counts on the queue vary from broker to broker even after the broker has had some time to settle.

      Working on a more consistent reproducer.

      Show
      We have been able to intermittently reproduce this by starting a 3-broker NOB and bringing up and down large numbers of consumers, waiting for them to balance and we see that the consumer counts on the queue vary from broker to broker even after the broker has had some time to settle. Working on a more consistent reproducer.

    Description

      In a network of brokers configured for balancing, similar to the attached configuration, when local consumers are added or removed, the network subscription count is not consistently reflected. It appears that both new and delete subscription advisories are sometimes either missed or suppressed with the result that not all live consumers get a remote consumer on the bridge and when all local consumers are shut down (no local consumers on any broker in the NOB), we still see consumers on the bridge, resulting in forwarding behavior when no actual demand exists on the other side of the bridge.

      This is occurring with no pendingMessageLimitStrategy applied. It seems to occur more frequently if suppressDuplicateQueueSubscriptions=true, but has been observed to occur without this flag set, as well.

      Attachments

        1. amq1logs120stuckconsumers.tar.gz
          269 kB
        2. amq1-log-stuck-NOB-consumer.tar.gz
          223 kB
        3. amq3logs120stuckconsumers.tar.gz
          139 kB
        4. amq3-log-stuck-NOB-consumer.tar.gz
          343 kB
        5. amq5logs120stuckconsumers.tar.gz
          137 kB
        6. ClientRebalanceAdvisoryTest.java
          6 kB
        7. ENTMQ-2195.diff
          30 kB
        8. ENTMQ-2195-TC1-more-logging.zip
          2.76 MB
        9. ENTMQ-2195-TC2-more-logging.zip
          10.39 MB
        10. logs-reproducer-no-suppress-amq1.tar.gz
          7.82 MB
        11. logs-reproducer-no-suppress-amq3.tar.gz
          7.70 MB
        12. logs-reproducer-no-suppress-amq5.tar.gz
          3.25 MB
        13. reproducer.xml
          6 kB
        14. reproducer-logs.zip
          471 kB
        15. stuck-consumers.txt
          30 kB
        16. stuckconsumers2nodesamq1.tar.gz
          2.35 MB
        17. stuckconsumers2nodesamq3.tar.gz
          74 kB
        18. stuckconsumers2nodesamq5.tar.gz
          2.34 MB
        19. threaded-mq-fabric-client.tar.gz
          26 kB

        Issue Links

          Activity

            People

              gtully@redhat.com Gary Tully
              rhn-support-dhawkins Duane Hawkins
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: