Search code examples
javaaws-lambdadroolsoptaplanner

Running OptaPlanner in AWS Lambda gives AlphaNetworkCompilation related error

I am attempting to run an Optaplanner (8.20.0) solver in AWS Lambda, uploaded in a fat-jar with all dependencies. When running a solution (Or more specifically when creating a solver: on SolverFactory.create(), I am encountering the following error:

  [ ... snip - large list of "Cannot find symbol" / "Cannot find package" errors ... ]
  location: class org.drools.ancompiler.CompiledPACKAGE_LOCATION_domain_ParticipationNetwork41408373267, org/drools/ancompiler/CompiledPACKAGE_LOCATION_domain_ParticipationNetwork41408373267.java (95:16) : cannot find symbol
  symbol:   variable objectTypeNode
  location: class org.drools.ancompiler.CompiledPACKAGE_LOCATION_domain_ParticipationNetwork41408373267, org/drools/ancompiler/CompiledPACKAGE_LOCATION_domain_ParticipationNetwork41408373267.java (99:16) : cannot find symbol
  symbol:   variable objectTypeNode
  location: class org.drools.ancompiler.CompiledPACKAGE_LOCATION_domain_ParticipationNetwork41408373267]
    at org.kie.memorycompiler.KieMemoryCompiler.compileNoLoad(KieMemoryCompiler.java:136)
    at org.kie.memorycompiler.KieMemoryCompiler.compileNoLoad(KieMemoryCompiler.java:104)
    at org.kie.memorycompiler.KieMemoryCompiler.compile(KieMemoryCompiler.java:56)
    at org.kie.memorycompiler.KieMemoryCompiler.compile(KieMemoryCompiler.java:42)
    at org.drools.ancompiler.KieBaseUpdaterANC.inMemoryUpdate(KieBaseUpdaterANC.java:66)
    at org.drools.ancompiler.KieBaseUpdaterANC.run(KieBaseUpdaterANC.java:52)
    at org.drools.ancompiler.KieBaseUpdaterANC.generateAndSetInMemoryANC(KieBaseUpdaterANC.java:99)
    at org.optaplanner.core.impl.score.director.stream.DroolsConstraintStreamScoreDirectorFactory.buildKieBaseFromModel(DroolsConstraintStreamScoreDirectorFactory.java:115)
    at org.optaplanner.core.impl.score.director.stream.DroolsConstraintStreamScoreDirectorFactory.buildKieBase(DroolsConstraintStreamScoreDirectorFactory.java:104)
    at org.optaplanner.core.impl.score.director.stream.DroolsConstraintStreamScoreDirectorFactory.<init>(DroolsConstraintStreamScoreDirectorFactory.java:71)
    at org.optaplanner.core.impl.score.director.stream.ConstraintStreamsScoreDirectorFactoryService.lambda$buildScoreDirectorFactory$0(ConstraintStreamsScoreDirectorFactoryService.java:50)
    at org.optaplanner.core.impl.score.director.ScoreDirectorFactoryFactory.decideMultipleScoreDirectorFactories(ScoreDirectorFactoryFactory.java:115)
    at org.optaplanner.core.impl.score.director.ScoreDirectorFactoryFactory.buildScoreDirectorFactory(ScoreDirectorFactoryFactory.java:52)
    at org.optaplanner.core.impl.solver.DefaultSolverFactory.buildScoreDirectorFactory(DefaultSolverFactory.java:168)
    at org.optaplanner.core.impl.solver.DefaultSolverFactory.<init>(DefaultSolverFactory.java:78)
    at org.optaplanner.core.api.solver.SolverFactory.create(SolverFactory.java:122)
[ ... snip - trace of my code before calling SolverFactory.create() ... ]

I seem to have traced the problem to the in-memory Alpha Network Compiler present in Drools. Specifically, disabling DroolsAlphaNetworkCompilationEnabled as suggested in this answer solves my problem. However, the problem does not seem to be that AWS Lambda does not provide a Java compiler. For one thing the error in the linked issue is different, and secondly this question (unrelated to optaplanner) seems to suggest that AWS Lambda functions are able to compile classes with some runtime classpath hacks.

My questions:

  1. What is the performance impact of settings DroolsAlphaNetworkCompilationEnabled to false (which currently fixes my issue)?
  2. Is there a way to avoid in-memory DroolsAlphaNetworkCompilation? Digging into some source files led me to KieBaseUpdaterANC.java in drools alphanetwork compiler which supports a LOAD option and loading the compiled network from a KJar. However, DroolsConstraintStreamScoreDirectorFactory calls generateAndSetInMemoryANC, which forces in-memory compilation. This might be the same question as DROOLS-5990
  3. As a measure of last resort, is the classpath-compiler-hack as suggested in this answer in any way relevant or implementable in my optaplanner/drools use-case?
Solution

  • Wrt. 1: The performance impact is possibly considerable. We have seen performance decreases of up to 50 %, but YMMV. It will depend heavily on the problem being solved.

    Wrt. 2: You have been able to disable ANC through the solver config. That is an officially supported option and you do not need to look for any other.

    Wrt. 3: I do not see a way to hook into our own ANC with a hack such as the one mentioned. Maybe using Quarkus, which may do this compilation at project build-time and not at runtime, could solve your problem?

    With all of that said, this error should not be showing up in the first place. Let's see if the Drools folks have anything to say about that.