I am trying to bin a float number R and I want to obtain its weight wt in an unsigned 8 bit integer format:
float R;
float edgeVal = rintf(R); // round to the nearest edge
float dR = R - edgeVal + 0.5f; // distance to the edge with a half-pixel offset
unsigned char wt = static_cast<unsigned char>( dR*256.f )); // allowed range [0; 255]
At the edge of the interval where dR == 0.f, we will get wt=0, but at the other edge where dR == 1.f, we will get wt = 256 which will wrap around to 0, while we really want to get wt = 255 in this case. What would be the proper code to achieve it?
Here is a 2-bit example for the range of variables:
wt = |0|1|2|3|
^ ^
| |
dR = 0 1
If only the exact value of 1.0f is the issue, you might want to avoid getting this value for dR in the first place. I would suggest a tiny change to your logic, replacing rintf by floor whose behaviour is easily predictable in all cases independently of any external factors:
float Radj = R + 0.5f;
float edgeVal = floor(R); // now Radj - 1 < edgeVal <= Radj
float dR = Radj - edgeVal; // 0 <= dR < 1
Now the original
static_cast<unsigned char>( dR*256.f ));
never overflows to 0.
For a few test values, the original code gives
R edgeVal dR wt
0 0 0.5 128
0.25 0 0.75 192
0.5 0 1 0
0.75 1 0.25 64
1 1 0.5 128
1.25 1 0.75 192
1.5 2 0 0
1.75 2 0.25 64
2 2 0.5 128
while with floor the values of edgeVal and wt are consistent (at the cost of adjusting the former):
R edgeVal dR wt
0 0 0.5 128
0.25 0 0.75 192
0.5 1 0 0 // note the change here
0.75 1 0.25 64
1 1 0.5 128
1.25 1 0.75 192
1.5 2 0 0
1.75 2 0.25 64
2 2 0.5 128