Search code examples
droolsoptaplanner

Drools much slower than EasyScoreCalculator [OptaPlanner]

Isn't Drools score calculation supposed to be faster than EasyScoreCalculator ?

Depending on the problem size I am working on I notice that Drools is 2-5x slower than EasyScoreCalculator.

e.g. Drools

2019-09-04 04:53:47,681 [main] INFO  Solving started: time spent (86), best score (-306init/[0]hard/[0/0]soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
2019-09-04 04:53:50,263 [main] INFO  Construction Heuristic phase (0) ended: time spent (2671), best score ([0]hard/[9272/0]soft), score calculation speed (24149/sec), step total (306).
2019-09-04 04:54:02,592 [main] INFO  Local Search phase (1) ended: time spent (15000), best score ([0]hard/[9272/0]soft), score calculation speed (31552/sec), step total (38993).
2019-09-04 04:54:02,593 [main] INFO  Solving ended: time spent (15001), best score ([0]hard/[9272/0]soft), score calculation speed (30079/sec), phase total (2), environment mode (REPRODUCIBLE).

EasyScoreCalculator

2019-09-04 04:53:10,282 [main] INFO  Solving started: time spent (8), best score (-306init/[-306]hard/[0]soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
2019-09-04 04:53:10,972 [main] INFO  Construction Heuristic phase (0) ended: time spent (699), best score ([0]hard/[9272]soft), score calculation speed (90657/sec), step total (306).
2019-09-04 04:53:25,273 [main] INFO  Local Search phase (1) ended: time spent (15000), best score ([0]hard/[9272]soft), score calculation speed (109948/sec), step total (193419).
2019-09-04 04:53:25,273 [main] INFO  Solving ended: time spent (15000), best score ([0]hard/[9272]soft), score calculation speed (108976/sec), phase total (2), environment mode (REPRODUCIBLE).

The only Drools rule I am using in this test is as follows:

rule "Maximise number of picked up parcels"
    when
      $job: Job(vehicle != null && vehicle.getVehicleType() != VehicleType.DUMMY)
    then
    scoreHolder.addSoftConstraintMatch(kcontext, 0, $job.getNumberOfParcels());
end

In EasyScoreCalculator this looks like:

public void calculateScore(VehicleRoutingSolution solution) {

    int numberOfPickedUpParcels = 0;

    List<Job> jobsList = solution.getJobs();

    for (Job job : jobsList) {

      Vehicle vehicle = job.getVehicle();

      if (vehicle!= null) {
        if (VehicleType.DUMMY != vehicle.getVehicleType()) {
          numberOfPickedUpParcels += job.getNumberOfParcels();
        }

    }

    int[] constraints = new int[1];
    int[] objectives = new int[1];

    constraints[0] = 0;
    objectives[0] = numberOfPickedUpParcels;

    return BendableScore.of(constraints, objectives);
  }

Isn't this peculiar? Is it because my rule is inefficient or are there caveats when Drools score calculation will be slower than easy ?

EDIT: This is a pretty late edit, but here is some data to back this up. Benchmarked the VRP datasets from here on various datasets from the 200, 600 and 1000 sized problemsets. While the performance of EasyScoreCalculator crashes for larger instances it is still faster than Drools.

Does it maybe have something to do with the amount of shadow variables there are in the problem? I remember running more complicated problems with more constraint when Drools did outperform EasyScoreCalculator, but that problem was more sophisticated.

enter image description here

EDIT2: I have re-run the experiment using the same dataset as above. Somehow Drools scales better now. Not sure why it is faster this time around with no config changes.

enter image description here

Solution

  • Drools score calculation is incremental, so it scales much better than EasyScoreCalculator which is not incremental.

    However, Drools does come with a performance overhead, so for smaller datasets, that overhead could undo the scaling gain. Use optaplanner-benchmark to try 3 different datasets, doubling in size each time, and look at the performance graph in the benchmark report.