How to implement a Java 8 Stream fluent API and lazily computed

Question

I would like to figure out a simple implementation equivalent to Java 8 Stream that would allow me to explore the development of query algorithms lazily computed (such as map(), filter(), reduce(), etc). NOTE: It is not my goal to achieve a better solution than Stream. On the other hand my only goal is to understand Stream internals.

Yet, every implementation that I found is based on the Iterable<T>, such as the solutions presented in the following answers:

• How are lazy streams implemented in Java 8?
• streaming is not a container of data, but a container of logic
• Java8 internal iteration

Yet, I do not feel comfortable with any of these solutions because:

1. they are too verbose.
2. they are not flexible for new query methods. The inclusion of new query methods requires structural modifications.
3. Despite the query arguments, they do not take any advantage of new Java 8 features such as: first class functions or default methods.
4. none of them use the Spliterator<T> approach that is used on Stream<T>.

I know that Spliterator<T> was designed to allow partitioning and parallel processing, but I think that its unique iterator method (boolean tryAdvance(Consumer<t>)) could be exploited to new alternatives than those ones listed above. Moreover and as stated by Brian Goetz:

Spliterator is a better Iterator, even without parallelism. (They're also generally just easier to write and harder to get wrong.)

So, is it possible to develop a more readable, simpler, concise and flexible implementation of a query API lazily computed and based on the same principles of the Stream<T> (except the parallel processing part)?

If yes, how can you do it? I would like to see simpler implementations than those ones listed above and if possible taking advantage of new Java 8 features.

Requirements:

• Do not reuse existing methods from Java 8 API
• The parallel processing feature is out of the scope of this question.
• If possible, and better, do not use the Iterable<T> approach.

The reason of my question? I think the best approach to learn a query API such as Stream is trying to implement those same methods by myself. I have already done it successfully when I was learning .net Linq. Of course that I did not achieve a better implementation than Linq but that helped me to understand the internals part. So, I am trying to follow the same approach to learn Stream.

This is not so unusual. There are many workshops following this approach for other technologies, such as the functional-javascript-workshop, which most exercises ask for the implementation of existing methods such a: map(), filter(), reduce(), call(), bind(), etc…

Selected Answer: For now I considered Miguel Gamboa’s answer as my choice, instead of Tagir Valeev’s answer because the latter does not allow the implementaton of findAny() or findFirst() without completely traversing whole elements through the forEach() of dataSrc. However, I think that Tagir Valeev’s answer have other merits regarding the concise implementation of some intermediate operations and also on performance, since the forEach() based approach, reduces the overhead of the iteration code that mediates access to the data structure internals as cited by Brian Goetz on point 2 of its answer

Answer 1

Using functional style programming and taking advantage of Java 8 default methods we can achieve a short and clean solution of a query API lazily computed. For instance, checkout how you can easily implement map() and forEach() methods in type Queryable bellow and then you can use it like this:

List<String> data = Arrays.asList("functional", "super", "formula");
Queryable.of(data) // <=> data.stream().
     .map(String::length)
     .forEach(System.out::println);

If you replace the Queryable.of(dataSrc) call with dataSrc.stream()you will get the same result. The following example illustrates an implementation of map() and forEach() methods. Check for the complete solution and a more detailed description at Queryable repository.

UPDATED with @srborlongan comment. Changed forEach signature from forEach(Consumer<T>) to forEach(Consumer<? super T>) and changed of from of(Collection<T>) to of(Iterable<T>)

@FunctionalInterface
public interface Queryable<T>{

  abstract boolean tryAdvance(Consumer<? super T> action); // <=> Spliterator::tryAdvance

  static <T> boolean truth(Consumer<T> c, T item){
    c.accept(item);
    return true;
  }

  public static <T> Queryable<T> of(Iterable<T> data) {
    final Iterator<T> dataSrc = data.iterator();
    return action -> dataSrc.hasNext() && truth(action, dataSrc.next());
  }

  public default void forEach(Consumer<? super T> action) {
    while (tryAdvance(action)) { }
  }

  public default <R> Queryable<R> map(Function<T, R> mapper) {
    return action -> tryAdvance(item -> action.accept(mapper.apply(item)));
  }
}