After a quick search in draft N4296, I could not find any example of a name in the C++ Standard Library for which two possible spellings exists (BrE vs. AmE).
While this may even be intentional, I can imagine that at some point, if a graphics library will be standardized (and there seems to be some effort going in this direction), the choice between "colour" and "color" will have to be made.
Are there normative regulations or perhaps even informal criteria that are being used to decide which spelling to pick for names in the C++ Standard Library?
I'd expect that, if there were an official guideline, we should be able to find it in the Standard Library Guidelines on the committee's web site. Since there are a number of items regarding naming conventions and none of them mentions American versus British English, I can only conclude that such a decision has not been made yet.
As others have commented, there is prose and at least one function name in the standard that uses the American spelling so my bet would be that it would be preferred in case of doubt. From what I can tell, this is also what most other libraries do. The Boost Library Requirements and Guidelines, by the way, don't have anything to say about this issue either.