Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-172

PFC with temp queues does not recover after limits hit

    XMLWordPrintable

Details

    • Bug
    • Resolution: Obsolete
    • Major
    • None
    • None
    • None
    • None

    Description

      Actually found during kit testing of 08-04.

      1. Start fresh database. Using default broker config with limits changed, I used 50MB ( <tempUsage limit="50 mb"/>)
      2. start producer, i.e

      wildflower:example sjavurek$ ant producer -DparallelThreads=5 -DmessageSize=5120 -Dmax=10000
      Buildfile: /Users/sjavurek/Fuse/activemq/RC/apache-activemq-5.5.1-fuse-08-04/example/build.xml
      
      init:
      
      compile:
      
      producer:
           [echo] Running producer against server at $url = tcp://localhost:61616 for subject $subject = TEST.FOO
           [java] Connecting to URL: tcp://localhost:61616 (:)
           [java] Publishing a Message with size 5120 to queue: TEST.FOO
           [java] Using non-persistent messages
           [java] Sleeping between publish 0 ms
           [java] Running 5 parallel threads
           [java] [Thread-6] Sending message: 'Message: 0 sent at: Fri Aug 17 13:21:46 EDT 2012  ...'
           [java] [Thread-3] Sending message: 'Message: 0 sent at: Fri Aug 17 13:21:46 EDT 2012  ...'
      

      3. Carries on until PFC kicks in.
      4. I start a consumer to try to remove messages and it just hangs. Coincidentally my jconsole window looses it's view of all the attributes including when I click on the broker. Can't shut it down cleanly either as someone is hung up:

      INFO | The connection to 'tcp://127.0.0.1:63148' is taking a long time to shutdown.

      Appears to be the producer and I can kill it:

      etswildflower:~ sjavurek$ netstat -a |grep 63148
      tcp4   65536      0  Susan-Javureks-M.61616 Susan-Javureks-M.63148 ESTABLISHED
      tcp4       0  63889  Susan-Javureks-M.63148 Susan-Javureks-M.61616 ESTABLISHED
      wildflower:~ sjavurek$ netstat -a |grep 63148
      wildflower:~ sjavurek$ 
      

      But then broker just hangs. I eventually kill -9 it.

      The windows side exhibits but result is same, can't pull messages off the queue at this point. There are actually wire format errors:

      consumer:
           [echo] Running consumer against server at $url = tcp://localhost:61616 for subject $subject = TEST.FOO
           [java] Connecting to URL: tcp://localhost:61616 (:)
           [java] Consuming queue: TEST.FOO
           [java] Using a non-durable subscription
           [java] Running 1 parallel threads
           [java] javax.jms.JMSException: Wire format negotiation timeout: peer did not send his wire format.
           [java]     at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
           [java]     at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1316)
           [java]     at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1402)
           [java]     at org.apache.ac[Thread-2] Caught: javax.jms.JMSException: Wire format negotiation timeout: peer did not
       send his wire format.
           [java] tivemq.ActiveMQConnection.start(ActiveMQConnection.java:507)
           [java]     at ConsumerTool.run(Unknown Source)
           [java] Caused by: java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
           [java]     at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:98)
           [java]     at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
           [java]     at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
           [java]     at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
           [java]     at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1287)
           [java]     ... 3 more
           [java] All threads completed their work
      

      And the producer actually falls over:

      consumer:
           [echo] Running consumer against server at $url = tcp://localhost:61616 for subject $subject = TEST.FOO
           [java] Connecting to URL: tcp://localhost:61616 (:)
           [java] Consuming queue: TEST.FOO
           [java] Using a non-durable subscription
           [java] Running 1 parallel threads
           [java] javax.jms.JMSException: Wire format negotiation timeout: peer did not send his wire format.
           [java]     at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
           [java]     at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1316)
           [java]     at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1402)
           [java]     at org.apache.ac[Thread-2] Caught: javax.jms.JMSException: Wire format negotiation timeout: peer did not
       send his wire format.
           [java] tivemq.ActiveMQConnection.start(ActiveMQConnection.java:507)
           [java]     at ConsumerTool.run(Unknown Source)
           [java] Caused by: java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
           [java]     at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:98)
           [java]     at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
           [java]     at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
           [java]     at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
           [java]     at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1287)
           [java]     ... 3 more
           [java] All threads completed their work
      

      I had a _quick_ look in JIRA and didn't see anything.

      Note, didn't take any thread dumps or anything - really need to get to some other issues today but can circle back around later if need be.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhn-support-sjavurek Susan Javurek
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: