Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-4847

Improve indexing performance

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Obsolete
    • Critical
    • None
    • 7.0.0.CR1
    • Embedded Querying, Lucene Directory
    • None

    Description

      This JIRA is focused on optimizing performance of use case where the application uses short or none transactions - therefore, no batching can be applied - and then performs the query and expects that the result will be consistent (please, specify in documentation if any lag after cache.put() or tx.commit() is required and if the application can detect that the update has been applied).

      We are well aware (see BZ) that indexing can be implemented with very low overhead, but the throughput is thousands of times lower [1] (configuration in [2]) with distributed mode storing index in replicated cache, and about 4x slowdown when comparing replicated cache without indexing and indexing to NRT FS/RAM.

      [1] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-indexing/7/artifact/results/html/test_test.html
      [2] https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-query-indexing/7/artifact/results/html/index.html

      Attachments

        Issue Links

          is blocked by

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-6453 Optimize indexing of putAll operations

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • To Do

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4692 Optimize externalizer for FileListCacheValue

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4694 Apply ASYNC backend indexing options to the RemoteIndexingBackend

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4697 Single cache operation generates two sequential synchronous calls to the indexing engine

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4708 Make DirectoryImplementor#deleteFile apply changes asynchronously

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4709 Have Query enable asynchronous delete operations for the Directory automatically

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4710 DistributedSegmentReadLocker should be allowed to skip ReadLocks on small files

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-5103 Inefficient index updates cause high cost merges and increase overall latency

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-4676 Optimize SearchFactoryHandler.updateKnownTypesIfNeeded to fast-path the most likely branch

          • Optional - The request should be considered desirable but is not an immediate necessity.
          • Closed
          is related to

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-3586 Improve performance of Infinispan Lucene directory provider

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Task - Represents a small unit of work that is not end-user facing. ISPN-5557 Core threading redesign

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          HSEARCH-1681 Loading...

          depends on

          HSEARCH-1693 Loading...

          HSEARCH-1697 Loading...

          HSEARCH-1698 Loading...

          HSEARCH-1699 Loading...

          Activity

            People

              gfernand@redhat.com Gustavo Fernandes (Inactive)
              rvansa1@redhat.com Radim Vansa (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: