Uploaded image for project: 'Application Server 7'
  1. Application Server 7
  2. AS7-5523

WritableServiceBasedNamingStore (jndi) installs a new msc service and waits for it, which could cause deadlocks

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Out of Date
    • Affects Version/s: 7.1.1.Final
    • Fix Version/s: None
    • Component/s: MSC, Server
    • Labels:
      None
    • Environment:

      windows 2008 server, single core VM

    • Workaround:
      Workaround Exists
    • Workaround Description:
      Hide

      To work around you can add -Dorg.jboss.server.bootstrap.maxThreads=20 (or some other larger number) just to make sure there are plenty of MSC threads.

      Show
      To work around you can add -Dorg.jboss.server.bootstrap.maxThreads=20 (or some other larger number) just to make sure there are plenty of MSC threads.

      Description

      class WritableServiceBasedNamingStore installs a new MSC service (instance of BinderService) and then waits for that MSC service to boot - by creating a new listener of type BindListener (an inner class) and calling listener.await() (line 72 in WritableServiceBasedNamingStore.java).

      i have a @Singleton @Startup EJB which, in its @PostConstruct method calls (new InitialContext().bind(...)), and hits this code.

      that @PostConstruct method was itself invoked by an MSC thread. the end result looks like this:

      "MSC service thread 1-2" prio=6 tid=0x000000000e146000 nid=0x960 in Object.wait() [0x000000000f6fd000]
      java.lang.Thread.State: WAITING (on object monitor)
      at java.lang.Object.wait(Native Method)

      • waiting on <0x00000007f652e400> (a org.jboss.as.naming.WritableServiceBasedNamingStore$BindListener)
        at java.lang.Object.wait(Object.java:503)
        at org.jboss.as.naming.WritableServiceBasedNamingStore$BindListener.await(WritableServiceBasedNamingStore.java:153)
      • locked <0x00000007f652e400> (a org.jboss.as.naming.WritableServiceBasedNamingStore$BindListener)
        at org.jboss.as.naming.WritableServiceBasedNamingStore.bind(WritableServiceBasedNamingStore.java:72)
        at org.jboss.as.naming.NamingContext.bind(NamingContext.java:221)
        at org.jboss.as.naming.InitialContext.bind(InitialContext.java:172)
        at org.jboss.as.naming.NamingContext.bind(NamingContext.java:230)
        at javax.naming.InitialContext.bind(Unknown Source)
        [application code ommitted]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:112)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228)
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333)
        at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
        at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161)
        at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85)
        at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116)
      • locked <0x000000078b4108e8> (a java.lang.Object)
        at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130)
        at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

      notice that this stack is during ComponentStartService.start(), which is in itself an MSC service. this is against the Service contract which states services should never block (or use the async api)

      this also has real world implications -

      the specific vm this was run on has a single (virtual) cpu, which means the MSC pool size is 2, one of which is now waiting.
      if two of my @Singletons happen to coincide the server deadlocks and will never finish booting (the thread dump excerpt above was taken just from such a situation)

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  dmlloyd David Lloyd
                  Reporter:
                  hatchetman82 Radai Rosenblatt
                • Votes:
                  6 Vote for this issue
                  Watchers:
                  8 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: