ArrayIndexOutOfBoundsException during generating benchmarking report in OptaPlanner

Question

So I want to run OptaPlanner's benchmarking and I have been able to succesfully implement this for various problems (tsp, vrp).

Now I am running it for a more complicated vrp, but I am being faced with an ArrayIndexOutOfBoundsException during the generation of the benchmark report.

// start benchmark
PlannerBenchmarkFactory plannerBenchmarkFactory = PlannerBenchmarkFactory.createFromXmlFile(
    new File("src/main/resources/vehiclerouting/benchmark/vrpBenchmarkConfig.xml"));

PlannerBenchmark plannerBenchmark = plannerBenchmarkFactory.buildPlannerBenchmark();
plannerBenchmark.benchmark(); //exception is thrown here.

Drilling down I found out it occurs in writeBestScoreSummaryChart() in BenchMarkReport.

private void writeBestScoreSummaryChart() {
    // Each scoreLevel has it's own dataset and chartFile
    List<DefaultCategoryDataset> datasetList = new ArrayList<>(CHARTED_SCORE_LEVEL_SIZE);
    for (SolverBenchmarkResult solverBenchmarkResult : plannerBenchmarkResult.getSolverBenchmarkResultList()) {
        String solverLabel = solverBenchmarkResult.getNameWithFavoriteSuffix();
        for (SingleBenchmarkResult singleBenchmarkResult : solverBenchmarkResult.getSingleBenchmarkResultList()) {
            String planningProblemLabel = singleBenchmarkResult.getProblemBenchmarkResult().getName();
            if (singleBenchmarkResult.hasAllSuccess()) {
                double[] levelValues = ScoreUtils.extractLevelDoubles(singleBenchmarkResult.getAverageScore());
                for (int i = 0; i < levelValues.length && i < CHARTED_SCORE_LEVEL_SIZE; i++) {
                    if (i >= datasetList.size()) {
                        datasetList.add(new DefaultCategoryDataset());
                    }
                    datasetList.get(i).addValue(levelValues[i], solverLabel, planningProblemLabel);
                }
            }
        }
    }
    bestScoreSummaryChartFileList = new ArrayList<>(datasetList.size());
    int scoreLevelIndex = 0;
    for (DefaultCategoryDataset dataset : datasetList) {
        String scoreLevelLabel = plannerBenchmarkResult.findScoreLevelLabel(scoreLevelIndex);
        CategoryPlot plot = createBarChartPlot(dataset,
                "Best " + scoreLevelLabel, NumberFormat.getInstance(locale));
        JFreeChart chart = new JFreeChart("Best " + scoreLevelLabel + " summary (higher is better)",
                JFreeChart.DEFAULT_TITLE_FONT, plot, true);
        bestScoreSummaryChartFileList.add(writeChartToImageFile(chart, "bestScoreSummaryLevel" + scoreLevelIndex));
        scoreLevelIndex++;
    }
}

In the above method datasetList reaches a size of 7. (We have 7 score levels -> 6 hard, 1 soft).

When plannerBenchmarkResult.findScoreLevelLabel(scoreLevelIndex); is called in the second loop findScoreLevelLabel() only returns 5 score levels. Namely: hard 0 score, hard 1 score hard 2 score hard 3 score and soft 0 score.

So once the scoreLevelLabel has a value >4 an ArrayIndexOutOfBoundsException is thrown. Any ideas / hints on why Optaplanner registers 5 score levels, whilst there should be 7.

EDIT Stacktrace:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 5
at org.optaplanner.benchmark.impl.result.PlannerBenchmarkResult.findScoreLevelLabel(PlannerBenchmarkResult.java:242)
at org.optaplanner.benchmark.impl.report.BenchmarkReport.writeBestScoreSummaryChart(BenchmarkReport.java:365)
at org.optaplanner.benchmark.impl.report.BenchmarkReport.writeReport(BenchmarkReport.java:241)
at org.optaplanner.benchmark.impl.DefaultPlannerBenchmark.benchmarkingEnded(DefaultPlannerBenchmark.java:306)
at org.optaplanner.benchmark.impl.DefaultPlannerBenchmark.benchmark(DefaultPlannerBenchmark.java:104)
at benchmark.BenchMarker.main(BenchMarker.java:57)

Answer 1

I have found the issue.

@PlanningScore(bendableHardLevelsSize = 4, bendableSoftLevelsSize = 1)
public BendableLongScore getScore() {
    return score;
}

The appropriate score levels were not updated when more levels were added. I don't know how a wrong declaration affects Optaplanner though. Are these levels used for other things other than (de)serialization ?