Search code examples
nservicebusnservicebus-distributor

NServiceBus Distributor - How to split application

We have 1 service which selects som id's from the database and then processes them with some buisness logic in sequence. The processing we would like to scale out and do in parallel, without creating a lot of internal threads.

My question is:

If I want to use the Distributor to scale out, how should and how can I do it?

Solution 1: The service is split into 2:

  1. The original service is modified to only select the ids from the database and then send them to the workers. This will be the distributor (RunDistributorWithNoWorkerOnItsEndpoint).
  2. A new service which will hold the buisness logic and function as a worker, processing every incomming id. If i want more workers i simple start the same process multiple times.

Solution 2: The service is split into 3:

  1. The original service is modified to only select the ids from the database and then send them to the distributor.
  2. A new service, the distributor (RunDistributorWithNoWorkerOnItsEndpoint), which only handles the load balancing to workers.
  3. A new service, worker(s), which will hold the buisness logic and process every incomming id.

Solution 3: The service is split into 2:

  1. The original service is modified to only select the ids from the database and then send them. Running as a sender.
  2. A new service which will hold the buisness logic and function as a worker and a distributor, processing every incomming id or distributing them.

Solution 1 makes a lot of sense to me, but using it would mean that somehow the distributor would have 2 responsabilities:

  1. Select the ids.
  2. Distribute the ids to workers.

But im not sure if this is possible or maybe even an antipattern, when I look at the ScaleOut example from the NSB documentation.

Solution 3 is what I believe I should go for, after reading the docs and the ScaleOut example againg but im not quite sure yet.

Im trying to solve a plumbing issue by scaling out with the NSB distributor and I would prefer to do my own hosting, not using the NServiceBus.Host.exe, but this is not a strict requirement.

EDIT: I ended up using solution 3 as it has the advantage (in my opinion) that the task of configuring queues is smaller than in solution 2, if you are use the default queue naming.

Kind regards

Solution

  • We've done this in the past versions using solution 2. The reason being is that we made the Distributor stand along and highly available. In the future we'll most likely let the Distributor do some of the work as otherwise it will just be sitting relatively idle. This solution has served us pretty well.