Uploaded image for project: 'AppFormer'
  1. AppFormer
  2. AF-918

Slow performance of VFS with GlusterFS

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • None
    • 2017 Week 49-50, 2017 Week 51-52, 2018 Week 01-02, 2018 Week 03-04, 2018 Week 05-06
    • NEW
    • NEW

      Any operation in business-central, eg. trying to access the Authoring -> Project Authoring or Administration option takes minutes to finish. When temporarily changing the location -Dorg.uberfire.nio.git.dir=/opt/data/jbpm-wb-git to local file system location other than gluster volume the response times were normal.

      The gluster log shows in a 70 second period that 175971 LOOKUPs are being done for files which don't exist:

      [root ~]# grep LOOKUP /var/log/glusterfs/bricks/jbpm-data.log | cut -f 2 -d P | cut -f 2 -d ' ' | grep -v scheduled | sort | uniq -c
        23844 /jbpm-wb-git/.niogit/my-poc.git/master
        23833 /jbpm-wb-git/.niogit/my-poc.git/refs/heads/master
        23844 /jbpm-wb-git/.niogit/my-poc.git/refs/master
        23840 /jbpm-wb-git/.niogit/my-poc.git/refs/tags/master
         6312 /jbpm-wb-git/.niogit/my-poc.git/shallow
          360 /jbpm-wb-git/.niogit/system.git/locks
        16770 /jbpm-wb-git/.niogit/system.git/master
          348 /jbpm-wb-git/.niogit/system.git/refs/da68722-uf-user
          360 /jbpm-wb-git/.niogit/system.git/refs/heads/locks
        16791 /jbpm-wb-git/.niogit/system.git/refs/heads/master
          360 /jbpm-wb-git/.niogit/system.git/refs/locks
        16773 /jbpm-wb-git/.niogit/system.git/refs/master
          360 /jbpm-wb-git/.niogit/system.git/refs/remotes
          360 /jbpm-wb-git/.niogit/system.git/refs/tags/locks
        16776 /jbpm-wb-git/.niogit/system.git/refs/tags/master
         4336 /jbpm-wb-git/.niogit/system.git/shallow
      [root ~]# grep LOOKUP /var/log/glusterfs/bricks/jbpm-data.log | cut -f 2 -d P | cut -f 2 -d ' ' | grep -v scheduled | sort | wc -l
      175971
      

      From the Gluster side, the recommended approach to address this is to enable "negative lookup caching" to prevent recurring access to files that do not exist to go through the whole filesystem.

      The aim of this ticket is to investigate if anything can be done from the uberfire / JGit side to prevent such a huge number of lookups to happen.

            eignatow Eder Ignatowicz
            abakos@redhat.com Alexandre Bakos
            Jan Hrcek Jan Hrcek (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: