According to the "Section 3 Summary" for Coursera's "Programming Languages, Part A" course (by Dan Grossman of the University of Washington):
But you have seen one feature that is more like dynamic scope than lexical scope: exception handling. When an exception is raised, evaluation has to “look up” which handle expression should be evaluated. This “look up” is done using the dynamic call stack, with no regard for the lexical structure of the program.
I think the writer is talking about Standard ML, but C++ seems also to do the same thing. Do all languages do "exception handling" with such dynamic lookups?
Yes, that is how exception handling works.
FWIW, exception handling as we understand it today was invented in the CLU language in the 70s and developed further in ML in the early 80s. From those it spread into other languages like C++, mostly only with variations on how exceptions are constructed and matched.
It is also worth noting that exception handling is just a special case of a more recently invented generalised mechanism called effect handlers, which is much richer and can express all sorts of other control structure, like coroutines, generators, async/await, even backtracking and more. Its main addition over exception handling is that a handler can resume the throwing computation, passing back a value. Like exception handling all its applications crucially rely on the dynamic extent of handlers.