The benefits to having a vector of reference-counted smart pointers are many. I no longer have to worry about cleaning up the memory, and I can even store pointers to derived classes in the container and they will be handled just fine also.
This is all really wonderful, but it makes me wonder about the implications. If my container is able to correctly clean up pointers to derived classes that I may have inserted into it, that means that a RTTI of some sort must occur. Would that cost be incurred even if I never place derived class pointers into the container, ever?
There is no RTTI, just straight-forward polymorphism. If your class hierarchy is already polymorphic (i.e. has virtual destructors, etc.), then there's absolutely no additional cost coming from the polymorphism.
What you should worry about is the cost of the shared pointer. If possible, see if a unique_ptr<Base> is actually enough. The atomic reference count update of the shared pointer is possibly a non-negligible cost.