Backport JBMESSAGING-1439 to EAP 4.3

Backport JBMESSAGING-1439 to EAP 4.3.

Here's a technical background of this request from the support case.

--------------------------------------------------------------------------------------------------------------------
ENVIRONMENT:

There are two nodes:

ESB node : SOA-P 4.3.0.GA + one-off of JBPAPP-2055
JMS node : SOA-P 4.3.0.GA + one-off of JBPAPP-2055

WS client(sync) (http)> EBWS on ESB node (jnp)> Queues on JMS node --> Postgres

The first arrow means SOAP request/response to EBWS and second one means
a jms-jca-provider[1] is defined on an ESB service, and looks up remote
JMS queues(request and reply for the ESB service) on JMS node via a JMS
external provider[2] defined on ESB.

[1] definition in jboss-esb.xml:

<jms-jca-provider name="JBossMessaging" connection-factory="XAConnectionFactory"
jndi-URL="jnp://localhost:1199"
providerAdapterJNDI="java:/remoteProvider">
<jms-bus busid="Test_ESB_JMS_XA_RecoverEsbChannel">
<jms-message-filter dest-type="QUEUE"
dest-name="queue/Test_ESB_JMS_XA_Recover_Queue" transacted="true" />
</jms-bus>
</jms-jca-provider>

[2] definition in -ds.xml:

<mbean code="org.jboss.jms.jndi.JMSProviderLoader"
name="jboss.esb:service=JMSProviderLoader,name=RemoteJMSProvider,server=remote">
<attribute name="ProviderName">remoteProvider</attribute>
<attribute name="ProviderAdapterClass">
org.jboss.jms.jndi.JNDIProviderAdapter
</attribute>
<attribute name="FactoryRef">XAConnectionFactory</attribute>
<attribute name="QueueFactoryRef">XAConnectionFactory</attribute>
<attribute name="TopicFactoryRef">XAConnectionFactory</attribute>
<attribute name="Properties">
java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
java.naming.factory.url.pkgs=org.jnp.interfaces
java.naming.provider.url=localhost:1199
</attribute>
</mbean>

DESCRIPTION:

Assume the two nodes run and an incoming request reaches from client to
a request queue of ESB service on JMS node via ESB node and a transaction
begins as jms-jca-provider is configured on the ESB service. Transaction
manager on ESB node creates a file(object store) for a transaction log
and is about to write the tx record in it.

At this point, you can see some records exist in jbm_tx table and via
jmx-console values of MessageCount and DeliveringCount attributes of the
request queue indicate positive while those of them are zero of the
reply queue as messages are transacted.

Then assume ESB node crashes(say kill -9) before writing any transaction
records in the created file. (See also JBTM-570.)

After the scenario above, ESB node boots again, then you'll see Ajruna's
transaction recovery module keeps logging a warn message with 2 minutes
interval in server log of ESB node. For example)

WARN [Thread-6 ] [arjLoggerI18N] [com.arjuna.ats.arjuna.recovery.RecoverAtomicAction_4] - RecoverAtomicAction: transaction not activated, unable to replay phase 2 commit

and you'll see that the records exist in jbm_tx table before the crash keep
remaining and never cleaned.