Affects Version/s: 5.4.2-fuse-03-00, 5.5.0-fuse-00-00
Java 6 (Sun JVM?)
Sun Solaris ?
ActiveMQ 5.4.2 (client library)
Java 1.5 (IBM JVM ?)
Sun Solaris ?
Similar Issues:Show 10 results
MB-378 Phantom consumer shows up in jconsole MB-188 support spool to disk for non-persistent topic consumers MB-221 Exclusive consumers are now selected up front when the consumer gets registered. MB-858 maven-activemq-perf-plugin cannot consume or produce any messages when using transactions MB-1202 data.db is not cleaned up properly with durable subscribers transacted MB-82 Misbehaving consumer can affect message delivery to other consumers for a given topic. MB-344 memoryLimit causes spooling to fail with persistent messages with no consumers. Queue fills up. MB-54 Exclusive consumers are now selected up front when the consumer gets registered. MB-545 messages left in JDBC database but not showing up in ActiveMQ web console nor JMX console MB-591 If tmp message store fills up, broker can deadlock due to while producers wait on disk space and consumers wait on acks
External Issue URL:
A customer is having an issue with an ActiveMQ deployment that is blocking their production roll-out. The deployment that causes trouble consists of:
- A producer application that reads many small messages from a large file and sends them to a queue. Each message is 2-3 kbytes long. The number of messages read from a file exceeds 100,000.
- An ActiveMQ broker that hosts the queue that receives those messages. That queue stores messages persistently. The broker is configured to use an amqPersistenceAdapter.
- The client is a Camel route inside a WebSphere container. The route performs business logic on each message received from the queue. The processing of a message takes roughly 1 second. The route uses Camel transactions to ensure complete processing of each message, and consumes messages from the queue in the transacted mode, 1 message per transaction.
To ensure processing of large files within reasonable time frames, the consumer is configured to use 80 threads. Initially, the threads were using a connection pool with maxConnections=10 and maximumActive=120. The customer claims that such configuration resulted in overall processing rate of 60 messages per second. However, messages were getting stuck in the pooled sessions after about 70,000 messages were processed.
I advised them to reduce the pool size to maxConnections=1 and maximumActive=80 (the number of threads) and set the pre-fetch limit to 1. That resolved the problem of stuck messages, but reduced the processing rate to about 13 messages per second. Thread dumps on the client show that most client threads are waiting for the broker to commit their transactions. Broker threads seem to be busy writing to disk. When they disable transactions, the processing rate goes back up, but they cannot do that in production.
Is it possible to tune the broker or client configuration to speed up transaction commits?