Abstracting repeated Intermediate Stream-operations into a single Function

Question

I am working with a large data structure where I want to perform a series of stream operations of the pattern:

<some stream>
    .map(<method reference getter that returns List>).filter(Objects::nonNull).flatMap(Collections::stream)
    .map(<method reference getter that returns another sub-List>).filter(Objects::nonNull).flatMap(Collections::stream)
    .forEach(<perform some operations>)

I would like to compose a definition that abstracts the map, filter, and flatMap operations into a single function that I can apply to the stream with a map or pass the stream to; in my head it looks something like this:

private static final <T,R> BiFunction<Stream<T>, Function<T,List<R>>, Stream<R>> mapAndFlatten =
    (inStream, lstGetter) -> {
        return inStream.map(lstGetter)
            .filter(Objects::nonNull)
            .flatmap(Collections::stream);
}

However, I'm not conceptualizing some things correctly. For one, the above syntax isn't right; is it obvious that I can't use generics with a BiFunction? Is there an existing framework/pattern to accomplish what I'm trying to do? Composing functions that make up subsequent map operations seems straight forward enough, so what about the addition of filter and flatMap is making it so hard for me to develop a solution? I'm struggling to find helpful information/examples. Am I conflating OO and functional concepts in a way that doesn't make sense?

Maybe I'm working too hard for a solution that doesn't need to exist; its not all that difficult to write .filter(Objects::nonNull).flatmap(Collections::stream), but it feels verbose.

Answer 1

it obvious that I cant use generics with a BiFunction?

You can use generics with functions. But you can't provide a function which consumes a stream as an argument of the map() operation, which expects an argument of type Function<? super T,? extends R>. I.e. consumes a stream element, not a stream. And not that expected type is Function, not BiFunction.

And since you're flattening the data, it would not work with map because map is meant to perform one-to-one transformations (meanwhile you have one-to-many transformation).

As the hosting operation for mapAndFlatten you need either flatMap() or mapMulti().

flatMap()

flatMap() expects a function consuming stream element and producing a stream of the resulting type Function<T, Stream<R>>.

That's how you can generate such function (Credits to @Holger for this concise implementation):

private static <T, R> Function<T, Stream<R>> mapAndFlatten(Function<T, List<R>> mapper) {
    
    return t -> Stream.ofNullable(mapper.apply(t))
        .flatMap(Collection::stream);
}

And the stream would look like this:

Stream.of(t1, t2, t3)
    .flatMap(mapAndFlatten(Method_Ref_1))
    .flatMap(mapAndFlatten(Method_Ref_2))
    .forEach(Perform_Some_Operations);

Dummy Example: a link to Online Demo

mapMulty()

Another option would be to make use of Java 16 mapMulti(). This operation expects an argument of type BiConsumer, which in turn consumes an element of the initial type and consumer of the resulting type. Every element fed to the consumer of the resulting type becomes a replacement of the initial element.

public static <T, R> BiConsumer<T, Consumer<R>> mapAndFlatten(Function<T, List<R>> mapper) {
    
    return (t, consumer) -> {
        List<R> list = mapper.apply(t);
        if (list != null) list.forEach(consumer);
    };
}

And the stream might look as follows:

Stream.of(t1, t2, t3)
    .<R1>mapMulti(mapAndFlatten(Method_Ref_1)) // with mapMulti() type inference mechanism would potentially require a hand in the form of type-witness <R>
    .<R2>mapMulti(mapAndFlatten(Method_Ref_2))
    .forEach(Perform_Some_Operations);

Dummy Example: a link to Online Demo

Note

• When you need to return a Collection, it's always a good idea to return an empty Collection instead of null. You might find this suggestion in many authoritative sources, such as "Effective Java" by Joshua Bloch. It makes interaction with the resulting value way simpler.