Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-10390

java.net.SocketTimeoutException: Read timed out on client when using scattered cache

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 13.0.0.Beta1
    • Clustering

    Description

      When running tests for RFE EAP7-867, the client is logging sampling errors due to
      java.net.SocketTimeoutExceptions:

      2018/04/20 06:54:59:695 EDT [WARN ][Runner - 647] HOST perf17.mw.lab.eng.bos.redhat.com:rootProcess:c - Error sampling data:  <org.jboss.smartfrog.loaddriver.RequestProcessingException: IO error: java.net.SocketTimeoutException: Read timed out>
              org.jboss.smartfrog.loaddriver.RequestProcessingException: IO error: java.net.SocketTimeoutException: Read timed out
      	at org.jboss.smartfrog.loaddriver.http.HttpRequestProcessorFactoryImpl$HttpRequestProcessor.processRequest(HttpRequestProcessorFactoryImpl.java:266)
      	at org.jboss.smartfrog.loaddriver.CompoundRequestProcessorFactoryImpl$CompoundRequestProcessor.processRequest(CompoundRequestProcessorFactoryImpl.java:52)
      	at org.jboss.smartfrog.loaddriver.Runner.run(Runner.java:103)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.net.SocketTimeoutException: Read timed out
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
      	at java.net.SocketInputStream.read(SocketInputStream.java:171)
      	at java.net.SocketInputStream.read(SocketInputStream.java:141)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
      	at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
      	at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
      	at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
      	at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
      	at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
      	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
      	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
      	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
      	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
      	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
      	at org.jboss.smartfrog.loaddriver.http.HttpRequestProcessorFactoryImpl$HttpRequestProcessor.processRequest(HttpRequestProcessorFactoryImpl.java:147)
      	... 3 more
      

      We have also tried to set bias-lifespan="0", but that did not help.
      There are two types of tests:

      1. failover tests
      Sampling errors occur after failover.

      run with bias-lifespan="0":
      overview: 1714 sampling errors
      http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-failover-http-session-shutdown-scattered-sync/17/artifact/report/failover.txt
      client log:
      http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-failover-http-session-shutdown-scattered-sync/17/console-perf17

      run where bias-lifespan is not set:
      overview: 620 sampling errors
      http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-failover-http-session-shutdown-scattered-sync/16/artifact/report/failover.txt
      client log:
      http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-failover-http-session-shutdown-scattered-sync/16/console-perf17/

      2. stress tests
      Sampling errors occur regularly, in larger numbers starting from 6th iteration till the end of the test.
      run with bias-lifespan="0":
      http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/15/
      run where bias-lifespan is not set:
      http://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/13/

      Performance comparison (available only from VPN):
      http://download.eng.brq.redhat.com/scratch/mvinkler/reports/2018-04-23_07-48-33/stress.html

      Also, WFLY-10273 is occurring in these runs.

      Attachments

        Activity

          People

            pferraro@redhat.com Paul Ferraro
            mvinkler1@redhat.com Michal Vinkler
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: