Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-3783

JBOSS AS Infinispan nodes cannot interoperate with Plain JVM / Glassfish nodes

    XMLWordPrintable

Details

    Description

      I am able to start up an Infinispan (running over JGroups 3.2.7.Final) cluster, with nodes composed of plain JVMs and Glassfish instances. Equally, I am able to create an Infinispan cluster composed purely of JBOSS instances.

      My problem occurs when I include a JBOSS instance in a cluster of Plain JVMs / Glassfish instances. Infinispan running on JBOSS does not appear to be able to inter-operate with Infinispan running in a different environment.

      The JGroups configuration is the same on all nodes and I am able to send messages between all of the nodes to prove that the JGroups cluster is operational.

      As far as I can see, the Infinispan configuration is the same across all nodes.

      The following exception is logged in the JBOSS log:
      2013-12-03 15:27:02,841 ERROR [org.jgroups.blocks.RequestCorrelator](OOB-18,shared=udp) failed unmarshalling buffer into return value: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 0
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:750)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
      at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:99)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:893)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1205)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
      at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:66)
      at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:58)
      at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:402)
      at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
      at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
      at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163)
      at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:116)
      at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104)
      at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
      at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:404)
      at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:249)
      at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:598)
      at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)
      at org.jgroups.JChannel.up(JChannel.java:707)
      at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
      at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
      at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
      at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
      at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
      at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
      at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:765)
      at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:420)
      at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:606)
      at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)
      at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
      at org.jgroups.protocols.FD.up(FD.java:253)
      at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
      at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)
      at org.jgroups.protocols.Discovery.up(Discovery.java:359)
      at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2616)
      at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
      at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
      at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1798)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:724)

      The following exception is logged on the plain JVM / Glassfish side:
      org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl
      .start() throws java.lang.Exception on object of type StateTransferManagerImpl
      at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
      at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:88
      3)
      at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:654)
      at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:643)
      at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:546)
      at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:199)
      at org.infinispan.CacheImpl.start(CacheImpl.java:559)
      at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:686)
      at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)
      at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:545)
      at com.saaconsultants.external.tester.DomainAccessTester.main(DomainAccessTester.java:137)
      Caused by: org.infinispan.CacheException: java.lang.IllegalArgumentException: Segment owner list cannot be null or empty

      at org.infinispan.util.Util.rewrapAsCacheException(Util.java:541)
      at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispat
      cher.java:186)
      at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:515)
      at org.infinispan.topology.LocalTopologyManagerImpl.executeOnCoordinator(LocalTopologyManagerImpl.java:261)
      at org.infinispan.topology.LocalTopologyManagerImpl.join(LocalTopologyManagerImpl.java:101)
      at org.infinispan.statetransfer.StateTransferManagerImpl.start(StateTransferManagerImpl.java:117)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
      ... 10 more
      Caused by: java.lang.IllegalArgumentException: Segment owner list cannot be null or empty
      at org.infinispan.distribution.ch.DefaultConsistentHash.<init>(DefaultConsistentHash.java:78)
      at org.infinispan.distribution.ch.DefaultConsistentHash$Externalizer.readObject(DefaultConsistentHash.java:314)
      at org.infinispan.distribution.ch.DefaultConsistentHash$Externalizer.readObject(DefaultConsistentHash.java:294)
      at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:402)
      at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
      at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
      at org.infinispan.topology.CacheTopology$Externalizer.readObject(CacheTopology.java:132)
      at org.infinispan.topology.CacheTopology$Externalizer.readObject(CacheTopology.java:121)
      at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:402)
      at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
      at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
      at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.readObject(SuccessfulResponse.java:101)
      at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.readObject(SuccessfulResponse.java:86)
      at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:402)
      at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:224)
      at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
      at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
      at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
      at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163
      )
      at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:116)
      at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:1
      04)
      at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
      at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:404)
      at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:249)
      at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:598)
      at org.jgroups.JChannel.up(JChannel.java:707)
      at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1020)
      at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
      at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
      at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
      at org.jgroups.protocols.pbcast.GMS.up(GMS.java:896)
      at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:245)
      at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:765)
      at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:420)
      at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:606)
      at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)
      at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
      at org.jgroups.protocols.FD.up(FD.java:253)
      at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288)
      at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)
      at org.jgroups.protocols.Discovery.up(Discovery.java:359)
      at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2616)
      at org.jgroups.protocols.TP.passMessageUp(TP.java:1263)
      at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1825)
      at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1798)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:724)

      I can supply my configuration if this would be useful.

      Attachments

        Activity

          People

            Unassigned Unassigned
            andrew_scully Andrew Scully (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: