Scalable workers in Azure Service Fabric

Question

I am trying to migrate to service fabric existing solution. Part of it is the number of workers that listen queues. For example I have some queues:

Task1_Queue, Task2_Queue ... TaskN_Queue

For each queue there is some logic of processing messages of them, let's say workers. They do different tasks like generate huge report files and upload to Azure Storage or do small updates in database.

The question is how to design services in order to have good scalability of workers. My thoughts were:

Option 1 - Each queue will have separate stateless service. No easy way to autoscale single service.

Option 2 - Implement workers as separate actors, and have single stateless service to listen queues and call actors. Pros - autoscaling out of box, as for each message from queue new actor will be created. Cons - actors will be disposable.

Answer 1

Consider creating 2 types of stateless services:

1. Service that monitors the task queue depth. This service will create and remove instances of Service 2 based on the amount of queued tasks.
2. Service that processes queued jobs.