Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 0.3
    • Fix Version/s: 0.4
    • Component/s: RDD
    • Labels:
      None

      Description

      When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
      This is caused by re-creating RemoteCachManager with pre-configured ISPN server address (for read here, for writes here), so when this server fails during RDD processing and Spark calls some function, which under the hood creates RemoteCacheManager, it will fail with connection refused exception.

      Here are some basic tests and example of exception thrown by HR client:

      org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
              at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
              at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
              at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
              at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
              at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
              at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
              at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
              at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
              at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
              at scala.Option.map(Option.scala:146)
              at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
              at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
              at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
              at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
              at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
              at org.apache.spark.scheduler.Task.run(Task.scala:89)
              at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
              at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
              at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
              at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
              ... 21 more
      Caused by: java.net.ConnectException: Connection refused
              at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
              at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
              at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
              ... 25 more
      

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  gustavonalle Gustavo Fernandes
                  Reporter:
                  vjuranek Vojtech Juranek
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: