Message Loss or premature execution termination in an action during shutdown

I'm able to reproduce JMS message loss or premature action termination during shutdown. I enqueued several thousand mesages into an ESB application. While the messages were being processed I toggle (stop/start) lifecycle state for the acting jms listeners via JMX in order to simulate shutdown. I noticed periodic warnings in the serverlog (listing 1). The final tally indicated message loss. Via code inspection the bug appears to be self-evident.

Basically when the doStop method (JMXConsole) is called it sets state to STOPPING and immediately terminates the executor thread (MessageAwareListener.java). If the thread was currently processing bad things happen.

I'm using transcated queues and it appears the JMS message transaction evaporates and the message is not returned to the queue. I added some code to AbstractThreadedManagedLifecycle.java to remedy the issue. It uses the default terminationPeriod (60 seconds), unless it is specified in the jboss-esb.xml file.

AbstractThreadedManagedLifecycle.java
---------------------------------------
protected void doStop()
throws ManagedLifecycleException
{
runningLock.lock() ;
try
{
if (isRunning())

// Dave Siracusa -start
if (!waitUntilStopped())

// Dave Siracusa - end
}
finally

}
MessageAwareListener.java
---------------------------
protected void doStop()
throws ManagedLifecycleException

Listing 1
----------
2010-09-01 15:04:40,924 WARN [org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle] (Thread-200) Unexpected error from doRun()
java.util.concurrent.RejectedExecutionException
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecutionThreadPoolExecutor.java:1768)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at org.jboss.soa.esb.listeners.message.MessageAwareListener.waitForEventAndProcess(MessageAwareListener.java:359)
at org.jboss.soa.esb.listeners.message.MessageAwareListener.doRun(MessageAwareListener.java:253)
at org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle.run(AbstractThreadedManagedLifecycle.java:115)
at java.lang.Thread.run(Thread.java:619)