Details

      Description

      Sometimes the ClientListenerNotifier keeps printing extremely high number of warning and quickly fill GBs of disk space:

      11:28:16,805 WARN  [ClientListenerNotifier] (Client-Listener-aacb322ffc60405e) ISPN004039: Unable to complete reading event from server null
      java.nio.channels.IllegalBlockingModeException
      	at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
      	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
      	at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:179)
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:282)
      	at org.infinispan.client.hotrod^C.impl.protocol.Codec20.readEvent(Codec20.java:126)
      	at org.infinispan.client.hotrod.event.ClientListenerNotifier$EventDispatcher.run(ClientListenerNotifier.java:237)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      

      This can be reproduced when running the server test suite, more specifically the test ExampleConfigsIT. Right after the server is started the above warning is logged in a loop for the duration of the tests.

      I've started to observe this after commit (1), which changed the behaviour to retry on SocketTimeouts.
      Before the aforementioned commit, log were printing:

      10:49:56,361 ERROR [ClientListenerNotifier] (Client-Listener-b53963d5ab2b4234) ISPN004043: Unrecoverable error reading event from server null, exiting event reader thread
      org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:184)
              at org.infinispan.client.hotrod.impl.protocol.Codec20.readMagic(Codec20.java:282)
              at org.infinispan.client.hotrod.impl.protocol.Codec20.readEvent(Codec20.java:126)
              at org.infinispan.client.hotrod.event.ClientListenerNotifier$EventDispatcher.run(ClientListenerNotifier.java:236)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.net.SocketTimeoutException
              at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
              at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
              at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
              at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
              at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.readByte(TcpTransport.java:179)
              ... 8 more
      

      And thus the listener thread was exiting,

      (1) https://github.com/infinispan/infinispan/commit/1a03a0a84307ae8b95c06b398216eb36f6065f31

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                gustavonalle Gustavo Fernandes
                Reporter:
                gustavonalle Gustavo Fernandes
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: