I was debugging an issue and figured out that the following snippet(sanitized) was the reason for the issue.
uint64_t testflag = 0;
testflag &= ~(0x08ul); //bug
testflag &= ~(0x08l); //expected
I compared the generated assembly and see this
uint64_t testflag = 0;
804851e: c7 45 d8 00 00 00 00 movl $0x0,-0x28(%ebp)
8048525: c7 45 dc 00 00 00 00 movl $0x0,-0x24(%ebp)
testflag &= ~(0x08ul);
804852c: 83 65 d8 f7 andl $0xfffffff7,-0x28(%ebp)
8048530: 83 65 dc 00 andl $0x0,-0x24(%ebp)
testflag &= ~(0x08l);
8048534: 83 65 d8 f7 andl $0xfffffff7,-0x28(%ebp)
8048538: 83 65 dc ff andl $0xffffffff,-0x24(%ebp)
Why does the NOT operator of unsigned long cause the compiler to AND 0 with the higher bytes instead of ffffffff.
My gcc version is gcc (GCC) 4.9.2 (Red Hat 4.9.2) on a 64bit machine.
Assuming 32-bit unsigned long/long...
uint64_t testflag;
0x08ul --> 00 00 00 08
~(0x08ul) --> FF FF FF F7
some_uint32_t = FF FF FF F7
testflag &= some_uint32_t;
testflag = testflag & some_uint32_t
testflag = testflag & (uint64_t) some_uint32_t
testflag = testflag & (uint64_t) FF FF FF F7 (or 4,294,967,288)
testflag = testflag & 00 00 00 00 FF FF FF F7
Converting a 32-bit unsigned to uint64_t is a simply 0 extend.
Now with ~(0x08l)
0x08l --> 00 00 00 08
~(0x08l) --> FF FF FF F7
some_int32_t = FF FF FF F7
testflag &= some_int32_t;
testflag = testflag & some_int32_t
testflag = testflag & (uint64_t) some_int32_t
testflag = testflag & (uint64_t) FF FF FF F7 (or - 8)
testflag = testflag & FF FF FF FF FF FF FF F7 (or 18,446,744,073,709,551,608)
-8 as a uint64_t is very nearly the maximum value for uint64_t.
With 2's complement, the result is the sign bit of OP's long is extended. @Marc Glisse