Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-5454

XSite: RetryMechanismTest random failures

    XMLWordPrintable

Details

    Description

      ClusteredCacheBackupReceiver.awaitRemoteTask() doesn't respect the state push command's timeout, at least when it's smaller than the sync replication timeout in the target cache. When that happens, the state provider will resend the state, and there will be 2 state push commands executing at the same time.

      RetryMechanismTest changes the state push timeout to 2 seconds, but the sync replication timeout stays at 15 seconds. This causes failures in testRetryLocally and testFailRetryLocally, if it takes more than 2 seconds to suspect the killed node.

      10:02:13,007 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)]
10:02:16,008 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)]
10:02:16,040 TRACE (asyncTransportThread-4,NodeP:) [RpcManagerImpl] replication exception: 
org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected
10:02:16,040 TRACE (asyncTransportThread-0,NodeP:) [RpcManagerImpl] replication exception: 
org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected
10:02:19,147 ERROR (testng-RetryMechanismTest:) [UnitTestTestNGListener] Test testFailRetryLocally(org.infinispan.xsite.statetransfer.failures.RetryMechanismTest) failed.
java.lang.AssertionError: expected:<2> but was:<3>
	at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
	at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364)
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80)
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245)
	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252)
	at org.infinispan.xsite.statetransfer.failures.RetryMechanismTest.testFailRetryLocally(RetryMechanismTest.java:227)

      Attachments

        Issue Links

          is related to

          Bug - A problem that impairs or prevents the functions of the product. JDG-648 XSite: RetryMechanismTest random failures

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-2338 XSite: RetryMechanismTest random failures

          • Optional - The request should be considered desirable but is not an immediate necessity.
          • Closed

          Activity

            People

              pruivo@redhat.com Pedro Ruivo
              dberinde@redhat.com Dan Berindei (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: