Uploaded image for project: 'Red Hat Decision Manager'
  1. Red Hat Decision Manager
  2. RHDM-71

Slow performance using business-central with GlusterFS

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • 7.1.0.GA
    • 6.x.x
    • Decision Central
    • None

    • Openshift image based on EAP 6.4, BPMS 6.4 and and a gluster volume mounted at /opt/data

    • CR1
    • ?

    Description

      Any operation in business-central, eg. trying to access the Authoring -> Project Authoring or Administration option takes minutes to finish. When temporarily changing the location -Dorg.uberfire.nio.git.dir=/opt/data/jbpm-wb-git to local file system location other than gluster volume the response times were normal.

      The gluster log shows in a 70 second period that 175971 LOOKUPs are being done for files which don't exist:

      [root ~]# grep LOOKUP /var/log/glusterfs/bricks/jbpm-data.log | cut -f 2 -d P | cut -f 2 -d ' ' | grep -v scheduled | sort | uniq -c
  23844 /jbpm-wb-git/.niogit/my-poc.git/master
  23833 /jbpm-wb-git/.niogit/my-poc.git/refs/heads/master
  23844 /jbpm-wb-git/.niogit/my-poc.git/refs/master
  23840 /jbpm-wb-git/.niogit/my-poc.git/refs/tags/master
   6312 /jbpm-wb-git/.niogit/my-poc.git/shallow
    360 /jbpm-wb-git/.niogit/system.git/locks
  16770 /jbpm-wb-git/.niogit/system.git/master
    348 /jbpm-wb-git/.niogit/system.git/refs/da68722-uf-user
    360 /jbpm-wb-git/.niogit/system.git/refs/heads/locks
  16791 /jbpm-wb-git/.niogit/system.git/refs/heads/master
    360 /jbpm-wb-git/.niogit/system.git/refs/locks
  16773 /jbpm-wb-git/.niogit/system.git/refs/master
    360 /jbpm-wb-git/.niogit/system.git/refs/remotes
    360 /jbpm-wb-git/.niogit/system.git/refs/tags/locks
  16776 /jbpm-wb-git/.niogit/system.git/refs/tags/master
   4336 /jbpm-wb-git/.niogit/system.git/shallow
[root ~]# grep LOOKUP /var/log/glusterfs/bricks/jbpm-data.log | cut -f 2 -d P | cut -f 2 -d ' ' | grep -v scheduled | sort | wc -l
175971

      From the Gluster side, the recommended approach to address this is to enable "negative lookup caching" to prevent recurring access to files that do not exist to go through the whole filesystem.

      The aim of this ticket is to investigate if anything can be done from the uberfire / JGit side to prevent such a huge number of lookups to happen.

      Attachments

        Issue Links

          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. AF-918 Slow performance of VFS with GlusterFS

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Open
          links to

          Web Link OpenShift Issue Report

          Activity

            People

              eignatow Eder Ignatowicz
              rhn-support-mputz Martin Weiler (Inactive)
              Karel Suta Karel Suta
              Karel Suta Karel Suta
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: