Let's say we have a graph that looks like this:
broadcast ~> flowA ~> fanIn
source ~> broadcast ~> flowB ~> fanIn ~> sink
broadcast ~> flowC ~> fanIn
Flows flowA, flowB,flowC all perform a transformation on the incoming elements. fanIn performs some combining action on the results of all three flows.
The challenge is that Flows A/B/C do not emit elements at the same rate. For some elements of the source, flowA has nothing to emit, while flowB and C continue emitting.
Now, at fanIn I want to be sure that the received elements on all three ports "belong" to the same element emitted from the source, i.e. that they are result of the transformations of the same element.
How would one go about this?
My current solution is to have Flows A/B/C emit Options. Each flow emits a Some if it can perform a transformation, and a None if it cannot. This way the number of emitted elements and rate on all three flows remains the same, and I can guarantee that received elements belong to the same source element. I'm looking for a more better performing solution that, if possible, does not require unnecessary object creation and wrapping.
Returning a None, will not really create a new object… Wraping to a Some (not to an Option which will do the nullcheck) can make better performance too. I think you can’t really bypass without a null element, but if you have a nonValid/null in your return type that can be used as None too. (For ex if they are objects with long id, you can create an invalid element with the id=-1 element and filter it out.) I think there will be no silver bullet here.
BUT: I think this is not a problem, you will not lost significant performance, your code will have probably other mutch larger bottlenecks, so let it go :D
(Reposted my answer from the lightbend discuss)