Uploaded image for project: 'Drools'
  1. Drools
  2. DROOLS-3535

Out of Memory Error when creating KieSession

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • None
    • 7.11.0.Final
    • core engine
    • None
    • 2019 Week 05-07
    • Hide

      Following pseudocode is given that can be used to reproduce this error:

      run loop for 2000 times:
      create session from drl rules

      Show
      Following pseudocode is given that can be used to reproduce this error: run loop for 2000 times: create session from drl rules
    • NEW
    • NEW

    Description

      We are using Drools compiler to compile DRL and create KieSession. Internally, when we try to compile large number of Spreadsheet (e.g. say 500), each containing about 30 rules, we get Out of memory error as shown in logs below:

      java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

      at java.base/java.lang.Thread.start0(Native Method)
      at java.base/java.lang.Thread.start(Thread.java:803)
      at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRulesLevel(KnowledgeBuilderImpl.java:1155)
      at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileRules(KnowledgeBuilderImpl.java:1135)
      at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.compileKnowledgePackages(KnowledgeBuilderImpl.java:973)
      at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildRules(KnowledgeBuilderImpl.java:2554)
      at org.drools.compiler.builder.impl.KnowledgeBuilderImpl.buildPackages(KnowledgeBuilderImpl.java:2489)
      at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.buildPackages(CompositeKnowledgeBuilderImpl.java:113)
      at org.drools.compiler.builder.impl.CompositeKnowledgeBuilderImpl.build(CompositeKnowledgeBuilderImpl.java:103)
      at org.drools.compiler.kie.builder.impl.AbstractKieProject.buildKnowledgePackages(AbstractKieProject.java:251)
      at org.drools.compiler.kie.builder.impl.AbstractKieProject.verify(AbstractKieProject.java:74)
      at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildKieProject(KieBuilderImpl.java:267)
      at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:235)
      at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:184)
      at org.kie.internal.utils.KieHelper.verify(KieHelper.java:94)
      at abc.xyz.KnowledgeBaseCreator.createKnowledgeBaseFromDrl(KnowledgeBaseCreator.java:319)

      This issue can be traced to KnowledgeBuilderImpl.java as shown below:

      Map<String, RuleBuildContext> ruleCxts = new ConcurrentHashMap<>();
      ForkJoinPool pool = new ForkJoinPool(); // avoid common pool
      try {
      pool.submit(() ->
      rules.stream().parallel()
      .filter(ruleDescr -> filterAccepts(ResourceChange.Type.RULE, ruleDescr.getNamespace(), ruleDescr.getName()))
      .forEach(ruleDescr -> {
      initRuleDescr(packageDescr, pkgRegistry, ruleDescr);
      RuleBuildContext context = buildRuleBuilderContext(pkgRegistry, ruleDescr);
      ruleCxts.put(ruleDescr.getName(), context);
      List<? extends KnowledgeBuilderResult> results = addRule(context);
      if (!results.isEmpty()) {
      synchronized (this.results)

      { this.results.addAll(results); }

      }
      })
      ).get();
      } catch (InterruptedException | ExecutionException e)

      { throw new RuntimeException("Rules compilation failed or interrupted", e); }

      for (RuleDescr ruleDescr : rules) {
      RuleBuildContext context = ruleCxts.get(ruleDescr.getName());
      if (context != null)

      { pkgRegistry.getPackage().addRule(context.getRule()); }

      }

      Here ForkJoinPool is created without specifying any parallelism limit as well as without shutting it down. Moreover, since its avoiding common pool, anyone who is using it in a loop will face the issue as discussed above.

      Attachments

        Issue Links

          Activity

            People

              mfusco@redhat.com Mario Fusco
              mesarvagya Sarvagya Pant (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: