from vs fromUnfiltered with nullable variable - Optaplanner

Question

I need clarification about the behaviour of from and fromUnfiltered when using planningEntities with nullable variables in Optaplanner.

The user documentation states that

The .from(T) building block selects every T instance that is in a problem fact collection or a planning entity collection and has no null planning variables.

To include instances with a null planning variable, especially if you use nullable variables, replace the from() building block by fromUnfiltered():

But the java doc of from methods states that from method will filter entities with null variables except if it's a nullable variable:

Start a ConstraintStream of all instances of the fromClass that are known as problem facts or planning entities.

If the fromClass is a PlanningEntity, then it will be automatically filtered to only contain fully initialized entities, for which each genuine PlanningVariable (of the fromClass or a superclass thereof) is initialized (so when the value is not null - unless PlanningVariable.nullable() is modified). This filtering will NOT automatically apply to genuine planning variables of subclass planning entities of the fromClass.

I'm not sure if my understanding is correct, and my tests show that javadoc is correct, and user documentation is misleading. Or perhaps I'm missing the point.

Anyone to clarify?

Answer 1

This whole situation with from(...) has been painful for a while. The behavior in OptaPlanner 8.13.0.Final and older is as follows:

• For entities with nullable = false planning variables (the default), from(...) returns only the entities that don't have null in any of their genuine planning variables.
• For entities with nullable = true planning variables (overconstrained planning), from(...) still returns entities with null in any of their genuine planning variables after these entities are intialized. In other words - it will not return them during construction heuristics, but it will return them in subsequent local search, if the construction heuristic has decided to keep the null there.

However, this is not the only confusing behavior when it comes to null handling. With conditional propagation (ifExists(...) etc.), we will always return entities with null genuine variables, in every situation.

We have recently realized that this situation is not sustainable and, starting with OptaPlanner 8.14.0.Final (to be released soon), we have deprecated from(...) and introduced forEach(...), which cleans up all this mess. From this version on, when using the new API, the behavior regarding null variables will be clearer and, more importantly, consistent.

forEach(...) will never return entities with null variables, forEachIncludingNullVars(...) should be used for that. And conditional propagation will honor that, too.

See upgrade recipe when the new version is out.