The Observer Pattern - further considerations and generalised C++ implementation

Question

A C++ MVC framework I’m writing makes heavy use of the observer pattern. I have had a thorough read of the related chapter in Design Patterns (GoF, 1995) and had a look at a multitude of implementations in articles and existing libraries (including Boost).

But as I was implementing the pattern, I could not help the feeling that there must be a better way - my client code involved lines and snippets that I felt should have been refactored into the pattern itself, if only I could find a way to overcome a few C++ limitations. Also, my syntax never appeared as elegant as that used in the ExtJs library:

// Subscribing
myGridPanel.on( 'render', this.onRender );

// Firing
this.fireEvent( 'render', null, node );

So I decided to conduct further research in attempt to arrive at a generalised implementation, while prioritizing code elegancy, readability and performance. I believe I've hit the jackpot on the 5th attempt.

The actual implementation, called gxObserver, is available on GitHub; it is well-document and the readme files spell out the pros as well as the cons. Its syntax is:

// Subscribing
mSubject->gxSubscribe( evAge, OnAgeChanged );

// Firing
Fire( evAge, 69 );

Having done what turned into an excessive work, I felt it would be just to share my findings with the SO community. So below I will answer this question:

What additional considerations (to these presented in Design Patterns) should programmers account for when implementing the observer pattern?

While focused around C++, many of the written below will apply in any language.

Please note: As SO limits answers to 30000 words, my answer had to be provided in 2 parts, but sometimes the second answer (the one starting with 'Subjects' shows up) first. Part 1 of the answer is the one starting with the class diagram from Design Patterns.

Answer 1

(start of part I)

Prerequisites

It’s Not All About State

Design Patterns ties the observer pattern to an object 'state'. As seen in the class diagram above (from Design Patterns), a subject’s state can be set using the SetState() method; upon state change the subject will notify all of its observers; then observers can inquire the new state using the GetState() method.

However, GetState() is not an actual method in subject base class. Instead, each concrete subject provides its own specialised state methods. An actual code might look like this:

SomeObserver::onUpdate( aScrollManager )
{
    // GetScrollPosition() is a specialised GetState();
    aScrollPosition = aScrollManager->GetScrollPosition();
}

What’s an object state? We define it as the collection of state variables – member variables that need to be persisted (for later reinstatement). For instance, both BorderWidth and FillColour could be state variables of a Figure class.

The idea that we can have more than one state variable – and thus an object’s state can change in more than one way – is important. It means that subjects are likely to fire more than one type of state change event. It also explains why it makes little sense to have a GetState() method in the subject base class.

But an observer pattern that can only handle state changes is an incomplete one – it is common for observers to observe stateless notifications, i.e., ones that are not state related. For example, the KeyPress or MouseMove OS events; or events like BeforeChildRemove, which clearly does not denote an actual state change. These stateless events are enough to justify a push mechanism – if observers cannot retrieve the change information from the subject, all the information will have to be served with the notification (more on this shortly).

There Will Be Many Events

It is easy to see how in 'real life' a subject may fire many types of events; a quick look at the ExtJs library will reveal that some classes offer upwards of 30 events. Thus, a generalised subject-observer protocol has to integrate what Design Patterns calls an 'interest' – allowing observers to subscribe to a particular event, and subjects to fire that event only to interested observers.

// A subscription with no interest.
aScrollManager->Subscribe( this );

// A subscription with an interest.
aScrollManager->Subscribe( this, "ScrollPositionChange" );

It Could Be Many-to-many

A single observer may observe the same event from a multitude of subjects (making the observer-subject relationship many-to-many). A property inspector, for instance, may listen to a change in the same property of many selected objects. If observers are interested in which subject sent the notification, the notification will have to incorporate the sender:

SomeSubject::AdjustBounds( aNewBounds )
{
    ...
    // The subject also sends a pointer to itself.
    Fire( "BoundsChanged", this, aNewBounds );
}

// And the observer receives it.
SomeObserver::OnBoundsChanged( aSender, aNewBounds )
{
}

It is worth noting, however, that in many cases observers don't care about the sender identity. For instance, when the subject is a singleton or when the observer’s handling of the event is not subject-dependant. So instead of forcing the sender to be part of a protocol we should allow it to be, leaving it to the programmer whether or not to spell the sender.

Observers

Event Handlers

The observer’s method which handles events (ie, the event handler) can come in two forms: overridden or arbitrary. Providing a critical and complex part in the implementation of the observers, the two are discussed in this section.

Overridden Handler

An overridden handler is the solution presented by Design Patterns. The base Subject class defines a virtual OnEvent() method, and subclasses override it:

class Observer
{
public:
    virtual void OnEvent( EventType aEventType, Subject* aSubject ) = 0;
};

class ConcreteObserver
{
    virtual void OnEvent( EventType aEventType, Subject* aSubject )
    {
    }
};

Note that we have already accounted for the idea that subjects typically fire more than one type of event. But handling all events (particularly if there are tens of them) in the OnEvent method is unwieldy - we can write better code if each event is handled in its own handler; effectively, this makes OnEvent an event router to other handlers:

void ConcreteObserver::OnEvent( EventType aEventType, Subject* aSubject )
{
    switch( aEventType )
    {
        case evSizeChanged:
            OnSizeChanged( aSubject );
            break;
        case evPositionChanged:
            OnPositionChanged( aSubject );
            break;
    }
}

void ConcreteObserver::OnSizeChanged( Subject* aSubject )
{
}

void ConcreteObserver::OnPositionChanged( Subject* aSubject )
{
}

The advantage in having an overridden (base class) handler is that it is dead easy to implement. An observer subscribing to a subject can do so by providing a reference to itself:

void ConcreteObserver::Hook()
{
    aSubject->Subscribe( evSizeChanged, this );
}

Then the subject just keeps a list of Observer objects and the firing code might look like so:

void Subject::Fire( aEventType )
{
    for ( /* each observer as aObserver */)
    {
        aObserver->OnEvent( aEventType, this );
    }
}

The disadvantage of overridden handler is that its signature is fixed, which makes the passing of extra parameters (in a push model) tricky. In addition, for each event the programmer has to maintain two bits of code: the router (OnEvent) and the actual handler (OnSizeChanged).

Arbitrary Handlers

The first step in overcoming the shortfalls of an overridden OnEvent handler is… by not having it all! It would be nice if we could tell the subject which method is to handle each event. Something like so:

void SomeClass::Hook()
{
    // A readable Subscribe( evSizeChanged, OnSizeChanged ) has to be written like this:
    aSubject->Subscribe( evSizeChanged, this, &ConcreteObserver::OnSizeChanged );
}

void SomeClass::OnSizeChanged( Subject* aSubject )
{
}

Notice that with this implementation we no longer need our class to inherit from the Observer class; in fact, we don’t need an Observer class at all. This idea is not a new one, it was described in length in Herb Sutter’s 2003 Dr Dobbs article called ‘Generalizing Observer’. But, the implementation of arbitrary callbacks in C++ is not a straightforward afair. Herb was using the function facility in his article, but unfortunately a key issue in his proposal was not fully resolved. The issue, and its solution are described below.

Since C++ does not provide native delegates, we need to use member function pointers (MFP). MFPs in C++ are class function pointers and not object function pointers, thus we had to provide the Subscribe method with both &ConcreteObserver::OnSizeChanged (The MFP) and this (the object instance). We will call this combination a delegate.

Member Function Pointer + Object Instance = Delegate

The implementation of the Subject class may rely on the ability to compare delegates. For instance, in cases we wish to fire an event to a specific delegate, or when we want to unsubscribe a specific delegate. If the handler is not a virtual one and belongs to the class subscribing (as opposed to a handler declared in a base class), delegates are likely to be comparable. But in most other cases the compiler or the complexity of the inheritance tree (virtual or multiple inheritance) will render them incomparable. Don Clugston has written a fantastic in-depth article on this problem, in which he also provides a C++ library that overcomes the problem; while not standard compliant, the library works with pretty much every compiler out there.

It is worth asking whether virtual event handlers are something we really need; that is, whether we may have a scenario where an observer subclass would like to override (or extend) the event handling behaviour of its (concrete observer) base class. Sadly, the answer is that this is a well possible. So a generalised observer implementation should allow virtual handlers, and we shall soon see an example of this.

The Update Protocol

Design Patterns’ implementation point 7 describes the pull vs push models. This section extends the discussion.

Pull

With the pull model, the subject sends minimal notification data and then the observer is required to retrieve further information from the subject.

We have already established that the pull model won’t work for stateless events such as BeforeChildRemove. It is perhaps also worth mentioning that with the pull model the programmer is required to add lines of code to each event handler that would not exist with the push model:

// Pull model
void SomeClass::OnSizeChanged( Subject* aSubject )
{
    // Annoying - I wish I didn't had to write this line.
    Size iSize = aSubject->GetSize();
}

// Push model
void SomeClass::OnSizeChanged( Subject* aSubject, Size aSize )
{
    // Nice! We already have the size.
}

Another thing worth remembering is that we can implement the pull model using a push model but not the other way around. Although the push model serves the observer with all the information it needs, a programmer may wish to send no information with specific events, and have the observers enquiring the subject for more information.

Fixed-arity Push

With a fixed-arity push model, the information a notification carries is delivered to the handler via an agreed amount and type of parameters. This is very easy to implement, but as different events will have a different amount of parameters, some workaround has to be found. The only workaround in this case would be to pack the event information into a structure (or a class) that is then delivered to the handler:

// The event base class
struct evEvent
{
};

// A concrete event
struct evSizeChanged : public evEvent
{
    // A constructor with all parameters specified.
    evSizeChanged( Figure *aSender, Size &aSize )
      : mSender( aSender ), mSize( aSize ) {}

    // A shorter constructor with only sender specified.
    evSizeChanged( Figure *aSender )
      : mSender( aSender )
    {
        mSize = aSender->GetSize();
    }

    Figure *mSender;
    Size    mSize;
};

// The observer's event handler, it uses the event base class.
void SomeObserver::OnSizeChanged( evEvent *aEvent )
{
    // We need to cast the event parameter to our derived event type.
    evSizeChanged *iEvent = static_cast<evSizeChanged*>(aEvent);

    // Now we can get the size.
    Size iSize  = iEvent->mSize;
}

Now although the protocol between the subject and its observers is simple, the actual implementation is rather lengthy. There are a few disadvantages to consider:

First, we need to write quite a lot of code (see evSizeChanged) for each event. A lot of code is bad.

Second, there are some design questions involved that are not easy to answer: shall we declare evSizeChanged alongside the Size class, or alongside the subject that fires it? If you think about it, neither is ideal. Then, will a size change notification always carry the same parameters, or would it be subject-dependent? (Answer: the latter is possible.)

Third, someone will need to create an instance of the event before firing, and delete it after. So either the subject code will look like this:

// Argh! 3 lines of code to fire an event.
evSizeChanged *iEvent = new evSizeChanged( this );
Fire( iEvent );
delete iEvent;

Or we do this:

// If you are a programmer looking at this line than just relax!
// Although you can't see it, the Fire method will delete this 
// event when it exits, so no memory leak!
// Yes, yes... I know, it's a bad programming practice, but it works.
// Oh.. and I'm not going to put such comment on every call to Fire(),
// I just hope this is the first Fire() you'll look at and just 
// remember.
Fire( new evSizeChanged( this ) );

Forth, there’s a casting business going on. We have done the casting within the handler, but it is also possible to do it within the subject’s Fire() method. But this will either involve dynamic casting (performance costly), or we do a static cast which could result in a catastrophe if the event being fired and the one the handler expects do not match.

Fifth, the handler arity is little readable:

// What's in aEvent? A programmer will have to look at the event class 
// itself to work this one out.
void SomeObserver::OnSizeChanged( evSizeChanged *aEvent )
{
}

As opposed to this:

void SomeObserver::OnSizeChanged( ZoomManager* aManager, Size aSize )
{
}

Which leads us to the next section.

Vari-arity Push

As far as looking at code goes, many programmers would like to see this subject code:

void Figure::AdjustBounds( Size &aSize )
{
     // Do something here.

     // Now fire
     Fire( evSizeChanged, this, aSize );
}

void Figure::Hide()
{
     // Do something here.

     // Now fire
     Fire( evVisibilityChanged, false );
}

And this observer code:

void SomeObserver::OnSizeChanged( Figure* aFigure, Size aSize )
{
}

void SomeObserver::OnVisibilityChanged( aIsVisible )
{
}

The subject’s Fire() methods and the observer handlers have different arity per event. The code is readable and as short as we could have hoped for.

This implementation involves a very clean client code, but would bring about a rather complex Subject code (with a multitude of function templates and possibly other goodies). This is a trade-off most programmers will take – it is better to have complex code in one place (the Subject class), than in many (the client code); and given that the subject class works immaculately, a programmer might just regard it as a black-box, caring little about how it is implemented.

What is worth considering is how and when to ensure that the Fire arity and the handler arity match. We could do it in run-time, and if the two don’t match we raise an assertion. But it would be really nice if we get an error during compile time, for which to work we’ll have to declare the arity of each event explicitly, something like so:

class Figure : public Composite, 
               public virtual Subject
{
public:
    // The DeclareEvent macro will store the arity somehow, which will
    // then be used by Subscribe() and Fire() to ensure arity match 
    // during compile time.
    DeclareEvent( evSizeChanged, Figure*, Size )
    DeclareEvent( evVisibilityChanged, bool )
};

We’ll see later how these event declaration have another important role.

(end of part I)