Search code examples
droolskie

Why Drools is executing incorrect rules when multiple KieBase are executed in parallel?

Drools seems to give incorrect results when multiple processes are run in parallel and in each process, a new KieBase object is created and processed every time.

Tried with versions: 6.5.0.Final, 7.32.0.Final

Details:

I executed 120 tasks in parallel (using 7 threads). Out of these 120 tasks, drools gave correct result for 108 tasks but executed incorrect rule for 12 tasks (no. of such failed tasks vary in each run).

Let me post the code and output here:

    public class TempClass {
        public List<String> droolLogging = new ArrayList<>();
    }

   public void execute(){
        Map<String, List<String>> failedTasks = new ConcurrentHashMap<>(); // to see which tasks were incorrectly executed

        // Run 120 tasks in parallel using x threads (x depends upon no of processor)

        IntStream.range(1, 120).parallel()
        .forEach(taskCounter -> {

            String uniqueId = "Task-"+taskCounter;

            TempClass classObj = new TempClass();

            String ruleString = "package com.sample" + taskCounter + "\n" +
                    "import com.TempClass\n" +
                    "\n" +\
                    "rule \"droolLogging"+taskCounter+"\"\n" +
                    "\t when \n" +
                    "\t\t obj: TempClass(true)\n" +
                    "\t then \n" +
                    "\t\t obj.droolLogging.add(\"RuleOf-"+uniqueId+"\");\n" +
                    "\t end\n";

            // Above ruleString contains 1 rule and it is always executed. 
            // After execution, it will add an entry in array list 'droolLogging' 
            // of class 'TempClass'. In this entry, we are storing task counter 
            // to see rule of which task is executed.


            //following line of code seems to be the culprit as this is somehow returning incorrect KieBase sometime.

            KieBase kbase = new KieHelper()
                             .addContent(ruleString, ResourceType.DRL)
                             .build();

        /*
        //Same issue occurs even if I create different file with different name instead of using KieHelper.

        KieServices ks = KieServices.Factory.get();
        KieFileSystem kfs = ks.newKieFileSystem();

        String inMemoryDrlFileName = "src/main/resources/inmemoryrules-" + taskCounter + ".drl";

        kfs.write(inMemoryDrlFileName, ruleString);

        KieBuilder kieBuilder = ks.newKieBuilder(kfs).buildAll();
        KieContainer kContainer = ks.newKieContainer(kieBuilder.getKieModule().getReleaseId());
        KieBaseConfiguration kbconf = ks.newKieBaseConfiguration();
        KieBase kbase = kContainer.newKieBase(kbconf);
        */


            StatelessKieSession kieSession = kbase.newStatelessKieSession();
            kieSession.execute(classObj);

            System.out.println("(" + Thread.currentThread().getName() + ") " +
                    uniqueId + "_" + classObj.droolLogging );

            //Important: 
            //  To see if correct rule is executed, task no. printed by variable 'droolLogging'
            //  should match with uniqueId

            if(classObj.droolLogging == null || classObj.droolLogging.size() != 1 ||
                    !classObj.droolLogging.get(0).endsWith(uniqueId)) {
                failedTasks.put("" + taskCounter, classObj.droolLogging);
            }
        });

        logger.info("Failed:\n {}", failedTasks);
    }


OUTPUT:
    (ForkJoinPool.commonPool-worker-1) Task-37_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-6) Task-8_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-3) Task-18_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-2) Task-108_[RuleOf-Task-4]
    (main) Task-78_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-7) Task-52_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-4) Task-97_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-5) Task-4_[RuleOf-Task-4]
    (ForkJoinPool.commonPool-worker-3) Task-19_[RuleOf-Task-19]
    (ForkJoinPool.commonPool-worker-5) Task-5_[RuleOf-Task-5]
    (ForkJoinPool.commonPool-worker-2) Task-109_[RuleOf-Task-109]
    (ForkJoinPool.commonPool-worker-7) Task-53_[RuleOf-Task-53]
    (ForkJoinPool.commonPool-worker-1) Task-38_[RuleOf-Task-38]
    (ForkJoinPool.commonPool-worker-4) Task-98_[RuleOf-Task-98]
    .... more

    Failed (12):
        {88=[RuleOf-Task-77], 78=[RuleOf-Task-4], 68=[RuleOf-Task-60], 37=[RuleOf-Task-4], 15=[RuleOf-Task-1], 18=[RuleOf-Task-4], 7=[RuleOf-Task-11], 8=[RuleOf-Task-4], 108=[RuleOf-Task-4], 71=[RuleOf-Task-76], 52=[RuleOf-Task-4], 97=[RuleOf-Task-4]}

This shows:

 - Rule of task 77 was executed in task 88
 - Rule of task 4 was executed in task 78
 - Rule of task 60 was executed in task 68
 - ....

This is wrong. For correct results, in each process, Rule of task X should be executed in task X only.

Any idea what can be the reason behind this ?

Update: Above code is just for testing purpose, to see how generation and execution of KieBase would behave in multi threaded environment. Actual use case is as follows:

Use Case:

We have a set of rules category wise. For each category, particular set of rules need to be executed.

Example:

 for category 1 , I need to execute rule101, rule102, rule103

 for category 2 , I need to execute rule201, rule202, rule203
 ....

 Note: During evaluation, rules of category X should NOT interfere with Rules of category Y, i.e., they should be run independently.

Since no. of categories are huge, we are building KieBases (for each category) in parallel and storing it for x minutes. After x minutes we check if rules have been changed for any category, if changed, KieBase is compiled again for those categories (which again would be in parallel).

Also, new categories can be added at run time. So, for newly added categories as well, above procedure is followed.

   category1 -> KieBase1   (compiled rules: rule101, rule102, rule103)
   category2 -> KieBase2   (compiled rules: rule201, rule202, rule203)
   category3 -> KieBase3

   Note: As already mentioned above, execution of KieBase X should NOT interfere with execution of KieBase Y as KieBases are created category wise and for each category, only particular set of rules should be executed.
Solution

  • It finally turns out to be a bug in Drools as KieHelper is not safe to be used in multi-threaded environment..

    After having suggested solution by one of the members of Drools Dev Community, following seems to be the reason for above mentioned anomaly and the workaround to overcome this issue:

    Root cause of issue : KieHelper builds a KieModule with the same default releaseId.

    Solution : Use different release ID for each build.

    Code: (use this in code mentioned above)

    KieServices ks = KieServices.Factory.get();
KieFileSystem kfs = ks.newKieFileSystem();
kfs.write("src/main/resources/rules.drl", ruleString);

ReleaseId releaseId = ks.newReleaseId("com.rule", "test" + taskCounter, "1.0.0");
kfs.generateAndWritePomXML(releaseId);

KieBuilder kieBuilder = ks.newKieBuilder(kfs).buildAll();
Results results = kieBuilder.getResults();

if (results.hasMessages(Message.Level.ERROR)) {
  throw new RuntimeException(results.getMessages().toString());
}

KieContainer kContainer = ks.newKieContainer(releaseId);
KieBase kbase = kContainer.newKieBase(ks.newKieBaseConfiguration());

StatelessKieSession kieSession = kbase.newStatelessKieSession();
kieSession.execute(classObj);

    Verified it by running >500 processes in parallel and issue didn't occur.