Obviously template libraries need to be header only, but for non-templates, when should you make things header-only?
If you think your non-template library could be header-only, consider dividing it into two files anyway, then providing a third file that includes both the .h and the .cpp (with an include guard).
Then anyone who uses your library in a lot of different TUs, and suspects that this might be costing a lot of compile time, can easily make the change to test it.
Once you know users have the option which way to use the library, the answer probably becomes "offer that option whenever you possibly can". So pretty much any time that including it from multiple TUs wouldn't violate the ODR. For instance, if your non-static free functions refer to static globals, then you're out of luck, since the different definitions of that function in different TUs would refer to different objects by the same name, which is an ODR-violation.