I have a collection of "business-process" documents and a separate collection of "business-process-events", each with a property business-process-id as a reference to the business-process of the event. Both documents are immutable to avoid concurrency issues, at least that is the idea so far.
I need to perform queries and show business-processes in a read-only "grid" with supplemental data based on a lot of logic from the business-process-events that has occured. Business-process-events can be stored out of order, especially
I thought about using map-reduce, but from past experience and available documentation, it will not be possible to do enough complex logic in the reduce.
If it was possible to subscribe to all new business-process-events and update a new index based on all business-process-events for the business-process at that point it would probably be a good solution, but how?
Instead of trying to solve this with an index, I dispatch a message (command) on a queue to perform the complex logic based on all the "events". This message is dispatched every time a new "event" occurs. Since the processing every time takes all "events" into consideration each time, the result is idempotent and even though there could be some duplicate processings, that is of no concern as long as the data is complete. We store the result of the complex logic processing in the "business-process" and make a simple map index on that.
We use NServiceBus as a messaging framework, but any kind of queueing can be used.