Search code examples
.nettask-parallel-librarytpl-dataflow

Is there a Dataflow block like TransformBlock<TIn, TOut> that allows items to be propagated out of sequence?


I'm relatively new to asynchronous processing and experimenting with TPL Dataflow. My scenario: I have a block that is continuously supplied with input, asynchronously executes a function on the input, and returns results. (The results are then passed to another block that saves to the database.) The function may complete within milliseconds or it could take as long as 10 seconds to return. The block is set up to process with "Unbounded" parallelism. The ordering of the results has no importance.

I started out using a TransformBlock but (because it maintains the item sequence) a single slow item would cause many faster items to pile up behind it. In this specific situation it is not only acceptable for the results to propagate out of sequence but highly desirable. This allows the results to flow at a regular rate, instead of in giant waves.

I've searched several times, looking for an implementation of a Dataflow block that propagates items as soon as they are complete, but I've yet to find anything that matches this description. I gave up and created my own NonSequentialBlock by "duct-taping" together an ActionBlock and a BufferBlock. It seems to do the job, but I'm concerned that (in my lack of experience) I've done something wrong and it will bite me in the end. Is there an existing implementation of this pattern available?


Solution

  • No, you haven't done anything wrong. It's typical to create your own blocks by combining the basic blocks, then calling Encapsulate to ... encapsulate them and present a single IPropagator block.