Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-1255

RequestIgnoredException on rehash using the Distributed Executor Service

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 5.0.0.FINAL
    • 5.0.0.CR7
    • None
    • None

    Description

      My application exposes its distributed operations via a REST-based infrastructure. To minimize the delta between JBoss starting and the cache starting, I used the new Distributed Executor to "sticky" a task to the data owner of a set of keys (with the same hash code).

      NOTE: Rehash still causes problems seen in ISPN-1106. (Attached new logs)

      I see a lot of the following error from the DistributedExecutorService when the new node's cache doesn't start in a timely manner:

      Reason: java.lang.IllegalStateException: Invalid response

      {Satriani-52149(PHL)=RequestIgnoredResponse}

      In addition, I see:
      org.infinispan.util.concurrent.TimeoutException: Timed out waiting for valid responses!

      It takes the cache about 2+ minutes at low throughput rate (30 tx/s) to recover. For high throughput rate, the cluster doesn't recover.

      Attachments

        1. cacheTest.zip
          33 kB
        2. ispn1255.log
          77 kB
        3. join_failure_thread_dump
          311 kB
        4. server_node1.log
          1.85 MB
        5. server_node2.log
          520 kB

        Activity

          People

            dberinde@redhat.com Dan Berindei (Inactive)
            esalter@synamedia.com Erik Salter (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: