I keep trying different search terms for this question and I'm just finding noise on both Google and stackoverflow. If I write code using C++'s standard library (std), is it all basically guaranteed to compile for Windows, Mac, and Linux (and hopefully work as intended)?
The standard defines what it means to be a C++ compiler, and all compilers claiming to be C++ should conform to the standard; any that don't can be considered buggy. All of the major compilers try their best to be conforming.
There are multiple standards to be concerned with here - C++98, C++03, C++11, C++14, C++17, and work has started on C++20. Sometimes the features in the latest current standard won't be implemented in every compiler. If you stick to C++03 you should find wide conformity.
Everything in the std namespace should be part of the standard, by definition.