I'm using a macro from a library supplied by NXP for their Arm M4F to convert from a float typed variable to a library data type called frac16_t and it doesn't work when the float value is less than -1. The frac16_t is typedef'd as a signed short.
The macro code is:
#define FRAC16(x) ((frac16_t)((x) < 0.999969482421875 ? ((x) >= -1 ? (x)*0x8000 : 0x8000) : 0x7fff))
The intended behavior of this macro is to convert any float value between -1 and +1 to a 16 signed integer representing the range from -1 to (1 - 1/2^15) with 0x8000 as the -1 value and 0x7fff and the nearly 1 value. If the float value is greater than (nearly) one the result saturates at 0x7fff and when the float is less than -1 the result is supposed to be 0x8000.
What actually happens is that for any input that's less than -1 the result is 0x7fff (i.e. nearly 1) and for any other value it works as advertised.
I did find that casting the 0x8000 constant to the frac16_t type makes the macro work correctly but I don't understand why the original library macro doesn't work. Changing the constant to -32768 also works and both of those fixes result in the constant being coded as 32 bits long which requires that the value be loaded indirectly from somewhere in flash near the load instruction vs loading as a 16 bit literal that's part of the instruction.
both of those fixes result in the constant being coded as 32 bits long which requires that the value be loaded indirectly from somewhere in flash
Not quite. The hex constants are converted to double and later to signed short.
First with test ? some_type_A : some_type_B, the result is a common type. In this case, double.
(x)*0x8000 is a double (or float), then : 0x8000 and then : 0x7fff also become the same floating point type.
0x8000 becomes a 32768.0. Assigning an out of range double to signed short is UB.
A common UB is out-of-range values take on the min/max limit.
In OP's case double 32768.0 became signed short 32767.
#define FRAC16(x) \\
((frac16_t)((x) < 0.999969482421875 ? ((x) >= -1 ? (x)*0x8000 : 0x8000) : 0x7fff))
// ^^^^^^
// 32768.0
Instead of assigning 32768.0 to a signed short and invoking undefined behavior, assign -32768.0 for defined behavior.
#define FRAC16(x)
((frac16_t)((x) < 0.999969482421875 ? ((x) >= -1 ? (x)*0x8000 : -32768 : 0x7fff))
// ^^^^^^^
// -32768.0
If one wants to code with SHRT_MIN, do not use 0x8000, use SHRT_MIN or (-0x7fff - 1).