Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-10391

ArrayIndexOutOfBoundsException on server using scattered cache

    Details

      Description

      We hit ArrayIndexOutOfBoundsException when running tests for RFE EAP7-867.

      EAP distribution was built from https://github.com/pferraro/wildfly/tree/scattered .
      Test description: Positive stress test (no failover), 4-node EAP cluster, clients: starting with 400 clients in the beginning, raising the number of clients to 6000 in the end of the test.

      Error occured on server dev215 around 7th iteration (can be seen in the performance report, link below):

      [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) Exception in thread "transport-thread--p15-t25" java.lang.ArrayIndexOutOfBoundsException: 129
      [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) 	at org.infinispan.scattered.impl.ScatteredVersionManagerImpl.lambda$tryRegularInvalidations$4(ScatteredVersionManagerImpl.java:413)
      [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) 	at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47)
      [JBossINF] [0m[31m04:26:11,708 ERROR [stderr] (transport-thread--p15-t25) 	at java.lang.Thread.run(Thread.java:748)
      

      Clients were getting "SocketTimeoutException: Read timed out" exceptions even before the ArrayIndexOutOfBoundsException ocurred, but also after.

      Performance report (accessible only when connected to VPN):
      http://download.eng.brq.redhat.com/scratch/mvinkler/reports/2018-04-19_15-10-14/stress.html

      One can observe that dev215 CPU usage and network usage dropped after 7th iteration.

      dev215 server log link:
      https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/eap-7x-stress-session-scattered-sync/13/console-dev215/

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  rhusar Radoslav Husar
                  Reporter:
                  mvinkler Michal Vinkler
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: