Interprocess messaging - MSMQ, Service Broker,?

Question

I'm in the planning stages of a .NET service which continually processes incoming messages, which involves various transformations, database inserts and updates, etc. As a whole, the service is huge and complicated, but the individual tasks it performs are small, simple, and well-defined.

For this reason, and in order to allow for easy expansion in future, I want to split the service into several smaller services which basically perform part of the processing before passing it onto the next service in the chain.

In order to achieve this, I need some kind of intermediary messaging system that will pass messages from one service to another. I want this to happen in such a way that if a link in the chain crashing or is taken offline briefly, the messages will begin to queue up and get processed once the destination comes back online.

I've always used message queuing for this type of thing, but have recently been made aware of SQL Service Broker which appears to do something similar. Is SQLSB a viable alternative for this scenario and, if so, would I see any performance benefits by using that instead of standard Message Queuing?

Thanks

Answer 1

If most of these steps initiate from a database state and end up in a database update, then merging your message storage with your data storage makes a lot of sense:

• a single product to backup/restore
• consistent state backups
• a single high-availability/disaster recoverability solution (DB mirroring, clustering, log shipping etc)
• database scale storage (IO capabilities, size and capacity limitations etc as per the database product characteristics, not the limits of message store products).
• a single product to tune, troubleshoot, administer

In addition there are also serious performance considerations, as having your message store be the same as the data store means you are not required to do two-phase commit on every message interaction. Using a separate message store requires you to enroll the message store and the data store in a distributed transaction (even if is on the same machine) which requires two-phase commit and is much slower than the single-phase commit of database alone transactions.

In addition using a message store in the database as opposed to an external one has advantages like queryability (run SELECT over the message queues).

Now if we translate the abstract terms 'message store in the database as being Service Broker and 'non-database message store' as being MSMQ, you can see my point why SSB will run circles any time around MSMQ.