Drools - Store Multi Stateful Sessions

Question

We have implemented drools engine in our platform in order to be able to evaluate rules from streams. In our use case we have a change detection stream which contains the changes of multiple entities.

Rules need to be evaluated for each entity from the stream over a period of time and evolve it's state apart from others entities(Sessions). Those rules produces alerts based on the state of each entity. And for this reason entities should be into boundaries, so the state of one entity does not interfere on the others.

To achieve this, we create a session as a Spring Bean for each entity id and store it in a inMemory HashMap. So every time an entity arrives, we try to find it`s session on the inMemory Map by using it's Id. If we get a null return we create it.

It does`t seems the right way to accomplish it. Because it does not offer a disaster recover strategy neither offers a great memory management.

We could use some kind of inMemory database such as Redis or Memchached. But I don`t think it would be able to recover a stateful session precisely.

Does someone know how to achieve disaster recover and a good memory management with a embedded Drools with multi sessions in the right way? Does the platform offers some solution?

Thanks very much for your attention and support

Answer 1

The answer is not to try to persist and reuse sessions, but rather to persist an object that models the current state of the entity.

Your current workflow is this:

1. Entity arrives at your application (from change detection stream or elsewhere)
2. You do a lookup on a hashmap to get a Session which has the entity's state stored
3. You fire the rules, which updates the session (and possibly the entity)
4. You persist the session in-memory.

What your workflow should be is this:

1. (same) Entity arrives at your application
2. You do a look-up on an external data source for the entity's state -- for example from a database or data store
3. You fire the rules, passing in the entity state. Instead of updating the session, you update the state instance.
4. You persist the state to your external data source.

If you add appropriate write-through caches you can guarantee both performance and consistency. This will also allow you to scale your application sideways if you implement appropriate locking / transaction handling for your data source.

---

Here's a toy example.

Let's say we have an application modelling a Library where a user is allowed to check out books. A user is only allowed to check out a total of 3 books at a time.

The 'event' we receive models a book check-in or check-out event:

class BookBorrowEvent {
  int userId;
  int bookId;
  EventType eventType; // EventType.CHECK_IN or EventType.CHECK_OUT
}

In an external data source we maintain a UserState record -- maybe as a distinct record in a traditional RDBMS or an aggregate; how we store it isn't really relevant to the example. But let's say our UserState record as returned from the data source looks something like this:

class UserState {
  int userId;
  int[] borrowedBookIds;
}

When we receive the event, we'll first retrieve the user state from the external data store (or an internally-managed write-through cache), then add the UserState to the rule inputs. We should be appropriately handling our sessions (disposing of them after use, using session pools as needed), of course.

public void handleBookBorrow(BookBorrowEvent event) {
  UserState state = getUserStateFromStore(event.getUserId());

  KieSession kieSession = ...; 
  kieSession.insert( event );
  kieSession.insert( state ); 
  kieSession.fireAllRules();

  persistUserStateToStore(state);
}

Your rules would then do their work against the UserState instance, instead of storing values in local variables.

Some example rules:

rule "User borrows a book"
when
  BookBorrowEvent( eventType == EventType.CHECK_OUT,
                   $bookId: bookId != null )
  $state: UserState( $checkedOutBooks: borrowedBookIds not contains $bookId )
  Integer( this < 3 ) from $checkedOutBooks.length
then
  modify( $state ) { ... }
end

rule "User returns a book"
when
  BookBorrowEvent( eventType == EventType.CHECK_IN,
                   $bookId: bookId != null )
  $state: UserState( $checkedOutBooks: borrowedBookIds contains $bookId )
then
  modify( $state ) { ... }
end

Obviously a toy example, but you could easily add additional rules for cases like user attempts to check out a duplicate copy of a book, user tries to return a book that they hadn't checked out, return an error if the user exceeds the 3 max book borrowing limit, add time-based logic for length of checkout allowed, etc.

Even if you were using stream-based processing so you can take advantage of the temporal operators, this workflow still works because you would be passing the state instance into the evaluation stream as you receive it. Of course in this case it would be more important to properly implement a write-through cache for performance reasons (unless your temporal operators are permissive enough to allow for some data source transaction latency). The only changes you need to make is to refocus your rules to target their data persistence to the state object instead of the session itself -- which isn't generally recommended anyway since sessions are designed to be disposed of.