Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-566

Master-slave replication does not seem to be properly synchronous

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • A-MQ 7.0.0.ER15
    • None
    • None
    • None
    • Hide

      1. On two machines (called "paddington" and "yogi" in my tests) install A-MQ ER12.
      2. Run "bin/artemis create temp" on each
      3. On the machine that is to be master ("paddington" in this case) replace the etc/broker.xml file with paddington.xml below.
      4. On the machine that is to be slave ("yogi" in this case) replace the etc/broker.xml file with yogi.xml
      5. Edit the two broker.xml files to have meaningful hostnames and/or port numbers, if necessary
      6. Start the two brokers
      7. Use any suitable JMS client to send a single message to any destination on the master broker, whilst monitoring the network traffic to the slave. The client should be capable of timing how long the send operation takes (I can supply a client if necessary). The contents of the message should appear on the wire between the master and slave – this is a way to be sure that the message has actually been forwarded
      8. Once messages are being forwarded correctly, add one second of delay to all network traffic between the master and the slave. It doesn't matter for this test if the delay is added to everything.

      1. tc qdisc change dev eth0 root netem delay 1000ms

      9. Check the delay is really working:

      kevin@paddington source]$ ping yogi
      PING yogi (192.168.1.70) 56(84) bytes of data.
      64 bytes from yogi (192.168.1.70): icmp_seq=1 ttl=64 time=1116 ms
      ...

      10. Retry the JMS message operation. If it takes less than two seconds (one second for a network packet in each direction), then the send on the master has returned to the client before the slave has received and confirmed the update.

      Show
      1. On two machines (called "paddington" and "yogi" in my tests) install A-MQ ER12. 2. Run "bin/artemis create temp" on each 3. On the machine that is to be master ("paddington" in this case) replace the etc/broker.xml file with paddington.xml below. 4. On the machine that is to be slave ("yogi" in this case) replace the etc/broker.xml file with yogi.xml 5. Edit the two broker.xml files to have meaningful hostnames and/or port numbers, if necessary 6. Start the two brokers 7. Use any suitable JMS client to send a single message to any destination on the master broker, whilst monitoring the network traffic to the slave. The client should be capable of timing how long the send operation takes (I can supply a client if necessary). The contents of the message should appear on the wire between the master and slave – this is a way to be sure that the message has actually been forwarded 8. Once messages are being forwarded correctly, add one second of delay to all network traffic between the master and the slave. It doesn't matter for this test if the delay is added to everything. tc qdisc change dev eth0 root netem delay 1000ms 9. Check the delay is really working: kevin@paddington source]$ ping yogi PING yogi (192.168.1.70) 56(84) bytes of data. 64 bytes from yogi (192.168.1.70): icmp_seq=1 ttl=64 time=1116 ms ... 10. Retry the JMS message operation. If it takes less than two seconds (one second for a network packet in each direction), then the send on the master has returned to the client before the slave has received and confirmed the update.

      When messages are produced to a broker that is the master of a master-slave pair, then the design intention is that the production operation will not complete until the slave broker confirms that it has the data. By deliberately slowing the network traffic between the master and the slave, I can see that the client's send operation complete successfully in a much shorter time that the round-trip network time between master and slave.

      I conclude, therefore, that the master is not waiting for the slave's response before acknowledging a successful operation to the client.

        1. paddington.xml
          3 kB
        2. yogi.xml
          3 kB

            rh-ee-ataylor Andy Taylor
            rhn-support-kboone Kevin Boone
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: