Uploaded image for project: 'FUSE ESB'
  1. FUSE ESB
  2. ESB-1447

Pooled Sessions are not being returned during the deployment of services.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: 3.3.1.3-fuse
    • Component/s: None
    • Labels:
      None

      Description

      There are some services that when you try to hot deploy them they generate a ClassNotFoundException for the referencing bean in the xbean.xml file for that service.

      Those same services will be deployed successfully when the system is restarted but this issue only happens when they are hot deployed.

      The process of deploying these services causes the pooled sessions to be borrow but they are not being released.

        Gliffy Diagrams

          Activity

          Hide
          ccorsi1 Claudio Corsi added a comment -

          The problem is that the pooled sessions are not being returned causing these sessions to be exhausted. Then the call to borrow another session caused the system to hang because the borrowObject call will cause the thread to wait until a session has been returned. Which never happens.

          Here is the stack trace that shows the issue:

          "Timer-3" daemon prio=3 tid=0x001c3800 nid=0x3d in Object.wait() [0xd3ffe000]
             java.lang.Thread.State: WAITING (on object monitor)
          	at java.lang.Object.wait(Native Method)
          	at java.lang.Object.wait(Object.java:485)
          	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:748)
          	- locked <0xdbe5a748> (a org.apache.commons.pool.impl.GenericObjectPool)
          	at org.apache.activemq.pool.SessionPool.borrowSession(SessionPool.java:54)
          	at org.apache.activemq.pool.ConnectionPool.createSession(ConnectionPool.java:118)
          	at org.apache.activemq.pool.PooledConnection.createSession(PooledConnection.java:141)
          	at org.apache.servicemix.jbi.nmr.flow.jms.AbstractJMSFlow.broadcast(AbstractJMSFlow.java:380)
          	at org.apache.servicemix.jbi.nmr.flow.jms.AbstractJMSFlow.onInternalEndpointRegistered(AbstractJMSFlow.java:354)
          	at org.apache.servicemix.jbi.nmr.flow.jms.AbstractJMSFlow$1.internalEndpointRegistered(AbstractJMSFlow.java:180)
          	at org.apache.servicemix.jbi.framework.EndpointRegistry.fireEvent(EndpointRegistry.java:526)
          	at org.apache.servicemix.jbi.framework.EndpointRegistry.registerInternalEndpoint(EndpointRegistry.java:215)
          	at org.apache.servicemix.jbi.framework.Registry.activateEndpoint(Registry.java:194)
          	at org.apache.servicemix.jbi.framework.ComponentContextImpl.activateEndpoint(ComponentContextImpl.java:126)
          	at org.apache.servicemix.common.endpoints.ProviderEndpoint.start(ProviderEndpoint.java:58)
          	at org.apache.servicemix.jms.endpoints.JmsProviderEndpoint.start(JmsProviderEndpoint.java:473)
          	- locked <0xfa414490> (a org.apache.servicemix.jms.endpoints.JmsProviderEndpoint)
          	at org.apache.servicemix.common.endpoints.SimpleEndpoint.activate(SimpleEndpoint.java:55)
          	- locked <0xfa414490> (a org.apache.servicemix.jms.endpoints.JmsProviderEndpoint)
          	at org.apache.servicemix.common.ServiceUnit.start(ServiceUnit.java:53)
          	at org.apache.servicemix.common.BaseServiceUnitManager.start(BaseServiceUnitManager.java:151)
          	- locked <0xdc0e9790> (a org.apache.servicemix.common.BaseServiceUnitManager)
          	at org.apache.servicemix.jbi.framework.ServiceUnitLifeCycle.start(ServiceUnitLifeCycle.java:103)
          	at org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:132)
          	- locked <0xfa471c98> (a org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle)
          	at org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:378)
          	at org.apache.servicemix.jbi.framework.AutoDeploymentService.updateServiceAssembly(AutoDeploymentService.java:355)
          	at org.apache.servicemix.jbi.framework.AutoDeploymentService.updateArchive(AutoDeploymentService.java:256)
          	at org.apache.servicemix.jbi.framework.AutoDeploymentService.monitorDirectory(AutoDeploymentService.java:664)
          	at org.apache.servicemix.jbi.framework.AutoDeploymentService.access$800(AutoDeploymentService.java:62)
          	at org.apache.servicemix.jbi.framework.AutoDeploymentService$1.run(AutoDeploymentService.java:628)
          	at java.util.TimerThread.mainLoop(Timer.java:512)
          	at java.util.TimerThread.run(Timer.java:462)

          The problem is this issue will exhaust all 500 pooled objects.

          Show
          ccorsi1 Claudio Corsi added a comment - The problem is that the pooled sessions are not being returned causing these sessions to be exhausted. Then the call to borrow another session caused the system to hang because the borrowObject call will cause the thread to wait until a session has been returned. Which never happens. Here is the stack trace that shows the issue: "Timer-3" daemon prio=3 tid=0x001c3800 nid=0x3d in Object.wait() [0xd3ffe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:485) at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:748) - locked <0xdbe5a748> (a org.apache.commons.pool.impl.GenericObjectPool) at org.apache.activemq.pool.SessionPool.borrowSession(SessionPool.java:54) at org.apache.activemq.pool.ConnectionPool.createSession(ConnectionPool.java:118) at org.apache.activemq.pool.PooledConnection.createSession(PooledConnection.java:141) at org.apache.servicemix.jbi.nmr.flow.jms.AbstractJMSFlow.broadcast(AbstractJMSFlow.java:380) at org.apache.servicemix.jbi.nmr.flow.jms.AbstractJMSFlow.onInternalEndpointRegistered(AbstractJMSFlow.java:354) at org.apache.servicemix.jbi.nmr.flow.jms.AbstractJMSFlow$1.internalEndpointRegistered(AbstractJMSFlow.java:180) at org.apache.servicemix.jbi.framework.EndpointRegistry.fireEvent(EndpointRegistry.java:526) at org.apache.servicemix.jbi.framework.EndpointRegistry.registerInternalEndpoint(EndpointRegistry.java:215) at org.apache.servicemix.jbi.framework.Registry.activateEndpoint(Registry.java:194) at org.apache.servicemix.jbi.framework.ComponentContextImpl.activateEndpoint(ComponentContextImpl.java:126) at org.apache.servicemix.common.endpoints.ProviderEndpoint.start(ProviderEndpoint.java:58) at org.apache.servicemix.jms.endpoints.JmsProviderEndpoint.start(JmsProviderEndpoint.java:473) - locked <0xfa414490> (a org.apache.servicemix.jms.endpoints.JmsProviderEndpoint) at org.apache.servicemix.common.endpoints.SimpleEndpoint.activate(SimpleEndpoint.java:55) - locked <0xfa414490> (a org.apache.servicemix.jms.endpoints.JmsProviderEndpoint) at org.apache.servicemix.common.ServiceUnit.start(ServiceUnit.java:53) at org.apache.servicemix.common.BaseServiceUnitManager.start(BaseServiceUnitManager.java:151) - locked <0xdc0e9790> (a org.apache.servicemix.common.BaseServiceUnitManager) at org.apache.servicemix.jbi.framework.ServiceUnitLifeCycle.start(ServiceUnitLifeCycle.java:103) at org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:132) - locked <0xfa471c98> (a org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle) at org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:378) at org.apache.servicemix.jbi.framework.AutoDeploymentService.updateServiceAssembly(AutoDeploymentService.java:355) at org.apache.servicemix.jbi.framework.AutoDeploymentService.updateArchive(AutoDeploymentService.java:256) at org.apache.servicemix.jbi.framework.AutoDeploymentService.monitorDirectory(AutoDeploymentService.java:664) at org.apache.servicemix.jbi.framework.AutoDeploymentService.access$800(AutoDeploymentService.java:62) at org.apache.servicemix.jbi.framework.AutoDeploymentService$1.run(AutoDeploymentService.java:628) at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462) The problem is this issue will exhaust all 500 pooled objects.
          Hide
          mhernon Matt Hernon added a comment -

          BQ:High

          Show
          mhernon Matt Hernon added a comment - BQ:High

            People

            • Assignee:
              Unassigned
              Reporter:
              ccorsi1 Claudio Corsi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: