-
Type:
Enhancement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: JBoss A-MQ 6.2, 6.1.1
-
Fix Version/s: JBoss A-MQ 6.3
-
Component/s: None
-
Labels:None
-
Environment:
JBoss A-MQ 6.1 broker using JDBC adapter for accessing Oracle.
A multi-threaded producer launches multiple threads that all concurrently send persistent messages to the same instance of a queue. The producer must use 'sync' sends to ensure guaranteed message delivery. These are the performance metrics, as provided by the requester, for different multi-threaded tests:| It takes 1 seconds for 1 thread producing 100 messages each : one thread writes one message in 0.01 seconds||It takes 2.5 seconds for 4 thread producing 100 messages each : one thread writes one message in 0.025 seconds|
| It takes 7 seconds for 10 threads producing 100 messages each : one thread writes one message in 0.07100 seconds |
| It takes 48 seconds for 100 threads producing 100 messages each : one thread writes one message in 0.48 seconds |
| It takes 60 seconds for 150 threads producing 100 messages each : one thread writes one message in 0.6 seconds |
| It takes 74 seconds for 200 threads producing 100 messages each : one thread writes one message in 0.74 seconds |
| It takes 108 seconds for 300 threads producing 100 messages each : one thread writes one message in 1.08 seconds |
A thread dump was taken during the 300 concurrent thread test, where it was noted that 299 NIO worker threads are all waiting on the sendLock within the Queue.doSendMessage() method (1): For example,| "ActiveMQ NIO Worker 634" daemon prio=10 tid=0x0000000062a30800 nid=0x70a4 waiting on condition[0x00002b56705d0000]||java.lang.Thread.State: WAITING (parking)|
| at sun.misc.Unsafe.park(Native Method) |
|
| at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) |
| at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) |
| at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(AbstractQueuedSynchronizer.java:867) |
| at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1201) |
| at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:312) |
| at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:827) |
| at org.apache.activemq.broker.region.Queue.send(Queue.java:727) |
| at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:390) |
Please refer to the attached thread dump.In this case, the message store is a DB; therefore, it is felt that synchronization should not be done within the doMessageSend() method and instead should be left up to the DBMS. This would then allow all 300 threads to concurrently access the DB and lets the DBMS control access to the relevant table(s).(1) https://github.com/jboss-fuse/activemq/blob/5.11.0.redhat-6-2-x-patch/activemq-broker/src/main/java/org/apache/activemq/broker/region/Queue.java#L816-L860
