Does KStream filter consume every message?

Question

I have used Kafka in the past, but never the streams API. I am tasked with building a scalable service that accepts websocket connections and routes outbound messages from a central topic to the correct session based on user id.

This looks ridiculously simple using KStream<String, Object>. From one online tutorial:

builder.stream(inputTopic, Consumed.with(Serdes.String(), publicationSerde))
        .filter((name, publication) -> "George R. R. Martin".equals(publication.getName()))
        .to(outputTopic, Produced.with(Serdes.String(), publicationSerde));

But does the filter command consume every message from the topic and perform a filter in application space? Or does KStream<K, V> filter(Predicate<? super K,? super V> predicate) contain hooks into the inner workings of Kafka that allow it only to receive messages matching the correct key?

The wording on the KStream<K,V> javadoc seem to suggest the former: "consumed message by message."

If the only purpose of the filter is to consume every message of a topic and throw away those that are not relevant, I could do that by hand.

Answer 1

You are correct - messages need to be deserialized, then inspected against a predicate (in application space)

throw away those that are not relevant, I could do that by hand

Sure, you could, but Kafka Streams has useful methods for defining session windows. Plus, you wouldn't need to define a consumer and producer instance to forward to new topics.