Uploaded image for project: 'Red Hat Decision Manager'
  1. Red Hat Decision Manager
  2. RHDM-404

Java-level deadlock after cloning a repository

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • 7.1.0.GA
    • 7.0.0, 7.0.0.GA
    • Decision Central
    • None
    • Release Notes
    • ER3
    • CR1

    Description

      I was cloning repositories, creating and deleting teams and suddenly Workbench UI stopped responding. I found out there is a number of threads in the BLOCKED state on the server.

      After cleaning up I wasn't able to get the UI completely stuck but in 100% of times there are 6 blocked threads and a deadlock reported by jps

      Found one Java-level deadlock:
      =============================
      "ForkJoinPool.commonPool-worker-7":
        waiting to lock monitor 0x00007f2bac09b6a8 (object 0x00000000c2ba92a8, a org.drools.core.common.ProjectClassLoader$DefaultInternalTypesClassLoader),
        which is held by "Thread-192"
      "Thread-192":
        waiting to lock monitor 0x00007f2bcc007828 (object 0x00000000c233c028, a org.drools.core.common.ProjectClassLoader),
        which is held by "ForkJoinPool.commonPool-worker-7"
      
      Java stack information for the threads listed above:
      ===================================================
      "ForkJoinPool.commonPool-worker-7":
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:404)
      	- waiting to lock <0x00000000c2ba92a8> (a org.drools.core.common.ProjectClassLoader$DefaultInternalTypesClassLoader)
      	at org.drools.core.common.ProjectClassLoader$DefaultInternalTypesClassLoader.loadType(ProjectClassLoader.java:428)
      	at org.drools.core.common.ProjectClassLoader.loadType(ProjectClassLoader.java:172)
      	at org.drools.core.common.ProjectClassLoader.loadClass(ProjectClassLoader.java:143)
      	- locked <0x00000000c233c028> (a org.drools.core.common.ProjectClassLoader)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	at org.drools.core.util.ClassUtils.safeLoadClass(ClassUtils.java:806)
      	at org.drools.core.base.ClassTypeResolver.resolveType(ClassTypeResolver.java:181)
      	at org.drools.core.base.ClassTypeResolver.resolveType(ClassTypeResolver.java:132)
      	at org.drools.compiler.rule.builder.PatternBuilder.getObjectType(PatternBuilder.java:325)
      	at org.drools.compiler.rule.builder.PatternBuilder.getObjectType(PatternBuilder.java:316)
      	at org.drools.compiler.rule.builder.PatternBuilder.build(PatternBuilder.java:176)
      	at org.drools.compiler.rule.builder.PatternBuilder.build(PatternBuilder.java:152)
      	at org.drools.compiler.rule.builder.PatternBuilder.build(PatternBuilder.java:133)
      	at org.drools.compiler.rule.builder.GroupElementBuilder.build(GroupElementBuilder.java:66)
      	at org.drools.compiler.rule.builder.RuleBuilder.build(RuleBuilder.java:105)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addRule(KnowledgeBuilderImpl.java:1205)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.lambda$compileRulesLevel$3(KnowledgeBuilderImpl.java:1166)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl$$Lambda$347/2081214425.accept(Unknown Source)
      	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
      	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
      	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
      	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
      	at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
      	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
      	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
      	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
      	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
      	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
      "Thread-192":
      	at org.drools.core.common.ProjectClassLoader$DefaultInternalTypesClassLoader.loadClass(ProjectClassLoader.java:419)
      	- waiting to lock <0x00000000c233c028> (a org.drools.core.common.ProjectClassLoader)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	at java.lang.Class.getDeclaredMethods0(Native Method)
      	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
      	at java.lang.Class.getDeclaredMethods(Class.java:1975)
      	at org.drools.compiler.builder.impl.TypeDeclarationCache.processModifiedProps(TypeDeclarationCache.java:243)
      	at org.drools.compiler.builder.impl.TypeDeclarationCache.initTypeDeclaration(TypeDeclarationCache.java:155)
      	at org.drools.compiler.builder.impl.TypeDeclarationCache.getAndRegisterTypeDeclaration(TypeDeclarationCache.java:91)
      	at org.drools.compiler.builder.impl.TypeDeclarationBuilder.getAndRegisterTypeDeclaration(TypeDeclarationBuilder.java:69)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.getAndRegisterTypeDeclaration(KnowledgeBuilderImpl.java:1784)
      	at org.drools.compiler.rule.builder.PatternBuilder.processClassObjectType(PatternBuilder.java:307)
      	at org.drools.compiler.rule.builder.PatternBuilder.build(PatternBuilder.java:182)
      	at org.drools.compiler.rule.builder.PatternBuilder.build(PatternBuilder.java:152)
      	at org.drools.compiler.rule.builder.PatternBuilder.build(PatternBuilder.java:133)
      	at org.drools.compiler.rule.builder.GroupElementBuilder.build(GroupElementBuilder.java:66)
      	at org.drools.compiler.rule.builder.RuleBuilder.build(RuleBuilder.java:105)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.addRule(KnowledgeBuilderImpl.java:1205)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.lambda$compileRulesLevel$3(KnowledgeBuilderImpl.java:1166)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl$$Lambda$347/2081214425.accept(Unknown Source)
      	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
      	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
      	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
      	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
      	at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
      	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
      	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
      	at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
      	at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
      	at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
      	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
      	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
      	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1162)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1142)
      	at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:979)
      	at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildRules(CompositeKnowledgeBuilderImpl.java:264)
      	at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:122)
      	at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:105)
      	at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:232)
      	at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
      	at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:251)
      	at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:219)
      	at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:182)
      	at org.kie.workbench.common.services.backend.builder.core.Builder.build(Builder.java:215)
      	- locked <0x00000000c2025c88> (a org.drools.compiler.kie.builder.impl.KieFileSystemImpl)
      	at org.kie.workbench.common.services.backend.builder.core.BuildHelper.build(BuildHelper.java:92)
      	at org.kie.workbench.common.services.backend.builder.core.BuildHelper$Proxy$_$$_WeldClientProxy.build(Unknown Source)
      	at org.kie.workbench.common.services.backend.builder.ala.LocalBuildExecConfigExecutor.apply(LocalBuildExecConfigExecutor.java:64)
      	at org.kie.workbench.common.services.backend.builder.ala.LocalBuildExecConfigExecutor.apply(LocalBuildExecConfigExecutor.java:32)
      	at org.kie.workbench.common.services.backend.builder.ala.LocalBuildExecConfigExecutor$Proxy$_$$_WeldClientProxy.apply(Unknown Source)
      	at org.guvnor.ala.pipeline.execution.PipelineExecutor.lambda$continuePipeline$0(PipelineExecutor.java:109)
      	at org.guvnor.ala.pipeline.execution.PipelineExecutor$$Lambda$305/1114939966.accept(Unknown Source)
      	at org.guvnor.ala.pipeline.StageUtil$1.execute(StageUtil.java:40)
      	at org.guvnor.ala.pipeline.StageUtil$1.execute(StageUtil.java:35)
      	at org.guvnor.ala.pipeline.execution.PipelineExecutor.continuePipeline(PipelineExecutor.java:94)
      	at org.guvnor.ala.pipeline.execution.PipelineExecutor.execute(PipelineExecutor.java:76)
      	at org.kie.workbench.common.services.backend.builder.ala.BuildPipelineInvoker.invokeLocalBuildPipeLine(BuildPipelineInvoker.java:84)
      	at org.kie.workbench.common.services.backend.builder.ala.BuildPipelineInvoker$Proxy$_$$_WeldClientProxy.invokeLocalBuildPipeLine(Unknown Source)
      	at org.kie.workbench.common.services.backend.builder.service.BuildServiceHelper.invokeLocalBuildPipeLine(BuildServiceHelper.java:152)
      	at org.kie.workbench.common.services.backend.builder.service.BuildServiceHelper.localBuild(BuildServiceHelper.java:83)
      	at org.kie.workbench.common.services.backend.builder.service.BuildServiceHelper$Proxy$_$$_WeldClientProxy.localBuild(Unknown Source)
      	at org.kie.workbench.common.services.backend.builder.service.BuildServiceImpl.build(BuildServiceImpl.java:67)
      	at org.kie.workbench.common.services.backend.builder.service.BuildServiceImpl$Proxy$_$$_WeldClientProxy.build(Unknown Source)
      	at org.kie.workbench.common.services.backend.builder.service.BuildInfoService.getBuildInfo(BuildInfoService.java:60)
      	at org.kie.workbench.common.services.backend.builder.service.BuildInfoService$Proxy$_$$_WeldClientProxy.getBuildInfo(Unknown Source)
      	at org.kie.workbench.common.services.backend.project.ProjectClassLoaderHelper.getProjectClassLoader(ProjectClassLoaderHelper.java:44)
      	at org.kie.workbench.common.services.backend.project.ProjectClassLoaderHelper$Proxy$_$$_WeldClientProxy.getProjectClassLoader(Unknown Source)
      	at org.kie.workbench.common.screens.datamodeller.backend.server.indexing.JavaFileIndexer.getProjectClassLoader(JavaFileIndexer.java:145)
      	at org.kie.workbench.common.screens.datamodeller.backend.server.indexing.JavaFileIndexer.fillIndexBuilder(JavaFileIndexer.java:129)
      	at org.kie.workbench.common.services.refactoring.backend.server.indexing.AbstractFileIndexer.toKObject(AbstractFileIndexer.java:78)
      	at org.kie.workbench.common.screens.datamodeller.backend.server.indexing.JavaFileIndexer$Proxy$_$$_WeldClientProxy.toKObject(Unknown Source)
      	at org.uberfire.ext.metadata.io.BatchIndex$3.visitFile(BatchIndex.java:178)
      	at org.uberfire.ext.metadata.io.BatchIndex$3.visitFile(BatchIndex.java:144)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:70)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:95)
      	at org.uberfire.java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:47)
      	at org.uberfire.java.nio.file.Files.walkFileTree(Files.java:1030)
      	at org.uberfire.java.nio.file.Files.walkFileTree(Files.java:1051)
      	at org.uberfire.ext.metadata.io.BatchIndex.run(BatchIndex.java:142)
      	at org.uberfire.ext.metadata.io.BatchIndex.run(BatchIndex.java:127)
      	at org.uberfire.ext.metadata.io.BatchIndex$1.run(BatchIndex.java:98)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      
      Found 1 deadlock.
      

      Attachments

        1. isolated1.zip
          819 kB
        2. jps.txt
          321 kB
        3. rhbxmsqe882.zip
          951 kB

        Issue Links

          Activity

            People

              eignatow Eder Ignatowicz
              rsynek@redhat.com Radovan Synek
              Radovan Synek Radovan Synek
              Radovan Synek Radovan Synek
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: