Uploaded image for project: 'ModeShape'
  1. ModeShape
  2. MODE-1876

OOM due to index rebuilding (if_missing) when starting repository with 1M nodes

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • 3.2.0.Final
    • 3.2.0.Final
    • JCR
    • None
    • Hide

      Setup the described configuration, and populate a repository with +1M nodes. Shutdown EAP 6. Start EAP 6 and trigger the startup of the repository (e.g. using the REST client). Wait and the OOM should occur.

      Show
      Setup the described configuration, and populate a repository with +1M nodes. Shutdown EAP 6. Start EAP 6 and trigger the startup of the repository (e.g. using the REST client). Wait and the OOM should occur.
    • Hide

      Disable index rebuilding.

      Show
      Disable index rebuilding.

      When attempting to open a repository with +1M indexed nodes, with indexing rebuild-upon-startup="if_missing" (the default), an OOM is encountered. This is with ISPN eviction strategy="LIRS" max-entries="50", and Java heap 4GB.

      This appears to be the result of the logic in org.modeshape.jcr.RepositoryQueryManager.reindexContent(boolean, boolean), which first reads the IDs of all 1M indexed node IDs into memory, and then crawls the entire repository in search of un-indexed nodes.

      I suspect that rebuild-upon-startup="always" would also fail, although I have not tried it.

      Also, note that in line 214 of RepositoryQueryManager.java, the entire contents of excludedKeysFromIndexing is logged (to debug). With millions of indexed nodes, this results in millions of IDs printed to the log, which is not desirable.

            rhauch Randall Hauch (Inactive)
            jonathandfields_jira Jonathan Fields (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: