Uploaded image for project: 'JBoss Enterprise Application Platform 4 and 5'
  1. JBoss Enterprise Application Platform 4 and 5
  2. JBPAPP-7464

JCA dead-lock when multiple threads execute org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.equals()

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • EAP_EWP 5.2.0
    • EAP_EWP 5.1.1
    • JCA
    • None
    • Release Notes
    • Hide
      When the <systemitem>org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.equals()</systemitem> method was executed simultaneously from multiple threads, a JCA-layer deadlock occurred. This update synchronizes three methods used within the <systemitem>equals()</systemitem> method, and changes the object type of the objects that were accessed on the deadlock from <systemitem>xaProps</systemitem> to <systemitem>HashMap</systemitem>. As a result, the <systemitem>equals()</systemitem> method can now be executed simultaneously by multiple threads and no deadlock occurs.
      Show
      When the <systemitem>org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.equals()</systemitem> method was executed simultaneously from multiple threads, a JCA-layer deadlock occurred. This update synchronizes three methods used within the <systemitem>equals()</systemitem> method, and changes the object type of the objects that were accessed on the deadlock from <systemitem>xaProps</systemitem> to <systemitem>HashMap</systemitem>. As a result, the <systemitem>equals()</systemitem> method can now be executed simultaneously by multiple threads and no deadlock occurs.
    • Documented as Resolved Issue
    • NEW

    Description

      Found one Java-level deadlock:
      =============================
      "http-10.61.0.11-20380-2":
      waiting to lock monitor 0x00002aaac57c4c08 (object 0x00000007807fdb98, a java.util.Properties),
      which is held by "Thread-278"
      "Thread-278":
      waiting to lock monitor 0x00002aaac428ddb8 (object 0x000000077bd422f0, a java.util.Properties),
      which is held by "JCA PoolFiller"
      "JCA PoolFiller":
      waiting to lock monitor 0x00002aaac57c4c08 (object 0x00000007807fdb98, a java.util.Properties),
      which is held by "Thread-278"

      Java stack information for the threads listed above:
      ===================================================
      "http-10.61.0.11-20380-2":
      at java.util.Hashtable.equals(Hashtable.java:736)

      • waiting to lock <0x00000007807fdb98> (a java.util.Properties)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.equals(XAManagedConnectionFactory.java:524)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory$SubjectActions.run(BaseWrapperManagedConnectionFactory.java:640)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory$SubjectActions.addMatchingProperties(BaseWrapperManagedConnectionFactory.java:655)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory.getConnectionProperties(BaseWrapperManagedConnectionFactory.java:504)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.getXAManagedConnection(XAManagedConnectionFactory.java:443)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.createManagedConnection(XAManagedConnectionFactory.java:409)
        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:639)
        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:273)
        at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:747)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:404)
        at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:424)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:496)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941)
        at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:89)
        at atg.service.jdbc.SwitchingDataSource.getConnection(SwitchingDataSource.java:908)
        at atg.adapter.gsa.GSATransaction.getConnection(GSATransaction.java:742)
      • locked <0x000000074ee06968> (a atg.adapter.gsa.GSATransaction)
        at atg.adapter.gsa.GSAItemDescriptor.getConnection(GSAItemDescriptor.java:2506)
        at atg.adapter.gsa.GSAItemDescriptor.executeCountQuery(GSAItemDescriptor.java:9439)
        at atg.adapter.gsa.GSAView.executeCountQuery(GSAView.java:672)
        at atg.adapter.gsa.GSAView.executeCountQuery(GSAView.java:622)
        at atg.multisite.SiteManager.getSiteCount(SiteManager.java:1007)
        at atg.multisite.session.MultisiteComponentSessionManager.hasSites(MultisiteComponentSessionManager.java:330)
        at atg.multisite.session.MultisiteComponentSessionManager.createSessionNameContext(MultisiteComponentSessionManager.java:184)
        at atg.servlet.NameContextManager.getSessionNameContext(NameContextManager.java:167)
      • locked <0x000000074ee02840> (a org.apache.catalina.session.StandardSessionFacade)
        at atg.servlet.ServletUtil.getSessionNameContext(ServletUtil.java:707)
        at atg.servlet.pipeline.HeadPipelineServlet.service(HeadPipelineServlet.java:800)
        at atg.servlet.pipeline.PipelineableServletImpl.service(PipelineableServletImpl.java:250)
        at atg.nucleus.servlet.NucleusProxyServlet.service(NucleusProxyServlet.java:215)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:183)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:95)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:451)
        at java.lang.Thread.run(Thread.java:662)

      "Thread-278":
      at java.util.Hashtable.size(Hashtable.java:206)

      • waiting to lock <0x000000077bd422f0> (a java.util.Properties)
        at java.util.Hashtable.equals(Hashtable.java:742)
      • locked <0x00000007807fdb98> (a java.util.Properties)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.equals(XAManagedConnectionFactory.java:524)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory$SubjectActions.run(BaseWrapperManagedConnectionFactory.java:640)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory$SubjectActions.addMatchingProperties(BaseWrapperManagedConnectionFactory.java:655)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory.getConnectionProperties(BaseWrapperManagedConnectionFactory.java:504)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.checkIdentity(BaseWrapperManagedConnection.java:522)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.getConnection(BaseWrapperManagedConnection.java:283)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:505)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941)
        at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:89)
        at atg.service.jdbc.SwitchingDataSource.getConnection(SwitchingDataSource.java:908)
        at atg.adapter.gsa.GSATransaction.getConnection(GSATransaction.java:742)
      • locked <0x00000007bc8ca5c8> (a atg.adapter.gsa.GSATransaction)
        at atg.adapter.gsa.GSAItemDescriptor.getConnection(GSAItemDescriptor.java:2506)
        at atg.adapter.gsa.GSAItemDescriptor.loadProperties(GSAItemDescriptor.java:5484)
        at atg.adapter.gsa.GSAItemDescriptor.loadProperty(GSAItemDescriptor.java:5610)
        at atg.adapter.gsa.GSAItem.getPersistentPropertyValue(GSAItem.java:1140)
        at atg.adapter.gsa.GSAItem.getPropertyValue(GSAItem.java:1033)
        at atg.adapter.gsa.GSAItem.getPropertyValue(GSAItem.java:1311)
        at atg.repository.RepositoryItemImpl.getPropertyValue(RepositoryItemImpl.java:154)
        at atg.repository.nucleus.RepositoryItemPropertyMapper.getPropertyValue(RepositoryItemPropertyMapper.java:129)
        at atg.beans.DynamicBeans.getPropertyValue(DynamicBeans.java:311)
        at atg.targeting.rules.PathResolver.resolveProperty(PathResolver.java:261)
        at atg.targeting.rules.PathResolver.resolvePath(PathResolver.java:223)
        at atg.targeting.rules.TargetRuleNode.evaluate(TargetRuleNode.java:192)
        at atg.targeting.rules.ComparisonRuleTree.evaluate(ComparisonRuleTree.java:238)
        at atg.targeting.rules.AndRuleTree.evaluate(AndRuleTree.java:182)
        at atg.targeting.rules.RuleNode.evaluate(RuleNode.java:293)
        at atg.targeting.rules.RuleNode.testTarget(RuleNode.java:349)
        at atg.targeting.RuleBasedRepositoryItemGroup.isGroupMember(RuleBasedRepositoryItemGroup.java:184)
        at atg.targeting.DynamicContentGroup.isGroupMember(DynamicContentGroup.java:610)
        at atg.repository.search.indexing.specifier.OutputItemSpecifier.isGroupMember(OutputItemSpecifier.java:1331)
        at atg.repository.search.indexing.IndexingOutputConfig.shouldIncludeItem(IndexingOutputConfig.java:2781)
        at atg.repository.search.indexing.LoaderImpl.processItem(LoaderImpl.java:458)
        at atg.repository.search.indexing.threading.ProcessItemJob.invoke(ProcessItemJob.java:123)
        at atg.common.util.ThreadDispatcherThread.run(ThreadDispatcherThread.java:147)

      "JCA PoolFiller":
      at java.util.Hashtable.get(Hashtable.java:333)

      • waiting to lock <0x00000007807fdb98> (a java.util.Properties)
        at java.util.Hashtable.equals(Hashtable.java:755)
      • locked <0x000000077bd422f0> (a java.util.Properties)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.equals(XAManagedConnectionFactory.java:524)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory$SubjectActions.run(BaseWrapperManagedConnectionFactory.java:640)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory$SubjectActions.addMatchingProperties(BaseWrapperManagedConnectionFactory.java:655)
        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnectionFactory.getConnectionProperties(BaseWrapperManagedConnectionFactory.java:504)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.getXAManagedConnection(XAManagedConnectionFactory.java:443)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.createManagedConnection(XAManagedConnectionFactory.java:409)
        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:639)
        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.fillToMin(InternalManagedConnectionPool.java:547)
        at org.jboss.resource.connectionmanager.PoolFiller.run(PoolFiller.java:74)
        at java.lang.Thread.run(Thread.java:662)

      Attachments

        Issue Links

          is related to

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) JBPAPP-10968 ManagedConnectionFactory bottleneck

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              jpederse@redhat.com Jesper Pedersen
              rhn-support-jbertram Justin Bertram
              Petr Penicka Petr Penicka (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: