I'm drawing a color selection button and I'm looking for a nice and simple formula to get a good text color (foreground) for a given background color in RGB.
A simple try would be to just take the complement color but this will produce an odd looking button for colors like pure blue or pure red.
Is there something well known that does this?
If it matters at all, I'm using QT.
For maximum legibility, you want maximum brightness contrast without getting into hues which don't work together. The most consistent way to do this is to stick with black or white for the text color. You might be able to come up with more aesthetically pleasing schemes, but none of them will be more legible.
To pick between black or white, you need to know the brightness of the background. This gets a little more complicated, due to two factors:
The perceived brightness of the individual primaries red, green, and blue are not identical. The quickest advice I can give is to use the traditional formula to convert RGB to gray - R*0.299 + G*0.587 + B*0.114. There are lots of other formulas.
The gamma curve applied to displays makes the middle gray value higher than you'd expect. This is easily solved by using 186 as the middle value rather than 128. Anything less than 186 should use white text, anything greater than 186 should use black text.