Is there a term for non-grayness of a color (where black and white would be non-gray)?
If the RGB space were rendered as a cube, white, black, red, green, blue, yellow, magenta, and cyan would be corners; gray would lie at the center of the cube.
Is there a name for colors on the surface of that cube?
In simple terms, non-grayness could be quantified (with r, g, and b from [0..1]) as
abs(max(r, g, b) - .5) / .5
Here, white, black, red, orange, etc. would have a "non-grayness" of 1.
A recent article in colour vision theory (an open-access version is available on bioRxiv) defined this concept as vividness.
It is based on the representation of colours in a colour solid, where each axis is one component of colour. In such solids, the origin is black and the opposite edge is white. For example, this is the colour solid of the CMYK colourspace, plotted in this StackOverflow question:
Vivid colours are colours on the surface of this colour solid, which is exactly what you are describing in your question:
We define a measure of colour vividness, such that points on the surface are maximally vivid and the ‘grey’ centre is minimally vivid.