Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-3727

Race condition during channel create vs start causing NPE on channel startup

    XMLWordPrintable

Details

    Description

      My hunch is that this is due to the fact that WildFly creates a channel in one thread but starts it in another. Specifically, Protocol.up_prot is set by ChannelService.start() (via new JChannel(...)) but is accessed by GlobalComponentRegistryService.start() (via Channel.connect(...)) - which happen in different threads. To prevents NPEs like the one below, we need to ensure that any variables set during channel creation are volatile.

      ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 30) MSC000001: Failed to start service jboss.infinispan.ejb.global-component-registry: org.jboss.msc.service.StartException in service jboss.infinispan.ejb.global-component-registry: org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on object of type ChannelTransport
          at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:91)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_51]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_51]
          at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]
          at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final.jar:2.1.1.Final]
      Caused by: org.infinispan.manager.EmbeddedCacheManagerStartupException: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on object of type ChannelTransport
          at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:241)
          at org.jboss.as.clustering.infinispan.subsystem.GlobalComponentRegistryService.start(GlobalComponentRegistryService.java:33)
          at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:86)
          ... 4 more
      Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.remoting.transport.jgroups.JGroupsTransport.start() on object of type ChannelTransport
          at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
          at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
          at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
          at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
          at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
          at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:219)
          ... 6 more
      Caused by: org.infinispan.commons.CacheException: Unable to start JGroups Channel
          at org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:198)
          at org.infinispan.remoting.transport.jgroups.JGroupsTransport.start(JGroupsTransport.java:187)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_51]
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_51]
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_51]
          at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_51]
          at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:183)
          ... 11 more
      Caused by: java.lang.NullPointerException
          at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1010)
          at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
          at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:391)
          at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:600)
          at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:147)
          at org.jgroups.protocols.FD.up(FD.java:255)
          at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:301)
          at org.jgroups.protocols.MERGE2.up(MERGE2.java:209)
          at org.jgroups.protocols.Discovery.up(Discovery.java:522)
          at org.jgroups.protocols.MPING.up(MPING.java:181)
          at org.jgroups.protocols.TP.fetchLocalAddresses(TP.java:2001)
          at org.jgroups.protocols.TP.start(TP.java:1131)
          at org.jgroups.protocols.TCP.start(TCP.java:82)
          at org.jgroups.stack.ProtocolStack.startStack(ProtocolStack.java:938)
          at org.jgroups.JChannel.startStack(JChannel.java:864)
          at org.jgroups.JChannel._preConnect(JChannel.java:527)
          at org.jgroups.JChannel.connect(JChannel.java:284)
          at org.jgroups.JChannel.connect(JChannel.java:275)
          at org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:196)
          ... 17 more
      

      Attachments

        Activity

          People

            pferraro@redhat.com Paul Ferraro
            pferraro@redhat.com Paul Ferraro
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: