Search code examples
c#azurepipelinedataflowazure-service-fabric

Service Fabric: Reliable Services pipeline with partitions load balancing


While trying to implement Service Fabric's Reliable Services pipeline, I had these three approaches to chose from:

enter image description here

And it looks like C is a good way to go. Details here.

In this case I need to implement kind of message pump between worker services.

For example I have 2 kinds of worker services. First one is IO-bound and scalability not required. Second is CPU-bound and scalability is required for it, so it uses a partitioning. I don't care what exactly partition will be used for process concrete item, so message pump must acts as a load balancer and enqueues item to CPU-bound service with minimum items in input queue. For now I've created a stateful service for this purpose.

In this form this looks very similar to TPL Dataflow pipeline.

My question is just am I using Service Fabric properly? Is there an overengineering here?

Do Reliable Actors fits better for this kind of pipelines? (or part of pipeline)

enter image description here


Solution

  • I do not think that Actors would be the right solution for this problem. The RunASync() method is hard to simulate in an Actor. You could use Timers and Reminders for that, but it feels unnatural. So I would go with a service for this one.