Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-2289

Cluster bridge Issue on server restart.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • AMQ 7.2.0.GA
    • clustering
    • None
    • Hide

      Start a 2 node cluster using static connectors.

            <connectors>
         <!-- Connector for the first broker in the cluster. -->
         <connector name="connector-broker001">tcp://10.0.0.161:59200</connector>
         <!-- Connector for the second broker in the cluster. -->
         <connector name="connector-broker002">tcp://10.0.0.165:59200</connector>
      </connectors>

      <cluster-connections>
         <cluster-connection name="cluster-DCA-APP">
            <address> </address>
            <connector-ref>connector-broker001</connector-ref>
            <retry-interval>500</retry-interval>
            <use-duplicate-detection>false</use-duplicate-detection>
            <message-load-balancing>ON_DEMAND</message-load-balancing>
            <max-hops>1</max-hops>
            <static-connectors>
               <connector-ref>connector-broker002</connector-ref>
            </static-connectors>
         </cluster-connection>
      </cluster-connections>

      Check netstat output:

      netstat -patn | fgrep 165 
tcp6       0      0 10.0.0.161:59200        10.0.0.165:38044        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:58618        10.0.0.165:59200        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:59200        10.0.0.165:38042        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:59200        10.0.0.165:38040        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:58622        10.0.0.165:59200        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:58620        10.0.0.165:59200        ESTABLISHED -

      Several connections are started between the nodes.

      Restart one of the nodes. Gracefully shut it down, and restart it. Check the network connections:

      [grovedc@t460p ~]$ netstat -patn | fgrep 165 
tcp6       0      0 10.0.0.161:58816        10.0.0.165:59200        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:58822        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:59200        10.0.0.165:38078        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:58618        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:58814        10.0.0.165:59200        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:59200        10.0.0.165:38080        ESTABLISHED -                   
tcp6       0      0 10.0.0.161:58622        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:58620        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:58812        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:58820        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:58818        10.0.0.165:59200        TIME_WAIT   -                   
tcp6       0      0 10.0.0.161:59200        10.0.0.165:38082        ESTABLISHED -

      There will be only 5 established connections and many connections in the time wait state.

      In this state, some message will be passed between the brokers, while others will not. This state requires that both brokers be restarted to clear this condition.

      Show
      Start a 2 node cluster using static connectors. <connectors> <!-- Connector for the first broker in the cluster. --> <connector name= "connector-broker001" >tcp: //10.0.0.161:59200</connector> <!-- Connector for the second broker in the cluster. --> <connector name= "connector-broker002" >tcp: //10.0.0.165:59200</connector> </connectors> <cluster-connections> <cluster-connection name= "cluster-DCA-APP" > <address> </address> <connector-ref>connector-broker001</connector-ref> <retry-interval>500</retry-interval> <use-duplicate-detection> false </use-duplicate-detection> <message-load-balancing>ON_DEMAND</message-load-balancing> <max-hops>1</max-hops> < static -connectors> <connector-ref>connector-broker002</connector-ref> </ static -connectors> </cluster-connection> </cluster-connections> Check netstat output: netstat -patn | fgrep 165 tcp6 0 0 10.0.0.161:59200 10.0.0.165:38044 ESTABLISHED - tcp6 0 0 10.0.0.161:58618 10.0.0.165:59200 ESTABLISHED - tcp6 0 0 10.0.0.161:59200 10.0.0.165:38042 ESTABLISHED - tcp6 0 0 10.0.0.161:59200 10.0.0.165:38040 ESTABLISHED - tcp6 0 0 10.0.0.161:58622 10.0.0.165:59200 ESTABLISHED - tcp6 0 0 10.0.0.161:58620 10.0.0.165:59200 ESTABLISHED - Several connections are started between the nodes. Restart one of the nodes. Gracefully shut it down, and restart it. Check the network connections: [grovedc@t460p ~]$ netstat -patn | fgrep 165 tcp6 0 0 10.0.0.161:58816 10.0.0.165:59200 ESTABLISHED - tcp6 0 0 10.0.0.161:58822 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:59200 10.0.0.165:38078 ESTABLISHED - tcp6 0 0 10.0.0.161:58618 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:58814 10.0.0.165:59200 ESTABLISHED - tcp6 0 0 10.0.0.161:59200 10.0.0.165:38080 ESTABLISHED - tcp6 0 0 10.0.0.161:58622 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:58620 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:58812 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:58820 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:58818 10.0.0.165:59200 TIME_WAIT - tcp6 0 0 10.0.0.161:59200 10.0.0.165:38082 ESTABLISHED - There will be only 5 established connections and many connections in the time wait state. In this state, some message will be passed between the brokers, while others will not. This state requires that both brokers be restarted to clear this condition.

      When a node in a cluster is restarted, the cluster bridge does not reconnect correctly.

        1. InfiniteReconnect.zip
          133 kB

            rhn-support-jbertram Justin Bertram
            rhn-support-dgrove Doug Grove (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: