Search code examples
javabytejavacunsigned-integer

How does masking work with int/Integer in java?

Since bytes are signed in java, in order to obtain unsigned integer values we mask with 0xFF asper:

byte signed  = -1;// -—> 0b11111111
int unsigned = signed & 0xFF;// —-> 0b11111111 & 0b11111111
System.out.println(“unsigned: ”+unsigned);//prints 255 instead of -1 since we masked earlier

But when the unsigned int is casted to a byte it seems nothing happens to it really i.e.

byte resigned = (byte) unsigned;
System.out.println(“resigned: ”+resigned);//same value as earlier signed = -1

Taking a closer look at the initial operation

int unsigned = signed & 0xFF;// the result of 0b11111111 & 0b11111111 is actually = 0b11111111

Even if the value of signed were something else say we had

byte  signed  = -44;// -—> 0b11010100
int  unsigned = signed & 0xFF;// the result of 0b11010100 & 0b11111111 is still actually = 0b11010100 
byte resigned = (byte) unsigned;
System.out.println(“resigned: ”+resigned);//same value as signed = -44

So it seems nothing really happens to the actual signed value for the unsigned int to be possible.

Does masking actually do anything to the (byte) value of an integer or it’s just a way to flag how javac should treat the integer?

Solution 
  • byte signed  = -1;// -—> 0b11111111

    Correct.

    int unsigned = signed & 0xFF;// —-> 0b11111111 & 0b11111111

    This is not correct. There are 3 steps.

    1. The value of signed is converted (sign-extended) to an int; i.e. 0b11111111 becomes 0b11111111111111111111111111111111 (= 0xFFFFFFFF)

    2. The 0b11111111111111111111111111111111 (= 0xFFFFFFFF) is ANDed with 0b00000000000000000000000011111111 (= 0x000000FF) to give 0b00000000000000000000000011111111 (= 0x000000FF)`

    3. 0b00000000000000000000000011111111 (= 0x000000FF) is assigned to unsigned.

    (Note: it is important to write this out in full so that you understand what happens.)

    byte resigned = (byte) unsigned;

    There are two steps here too.

    1. The cast (byte) converts 0b00000000000000000000000011111111 (= 0x000000FF) to 0b11111111 (= 0xFF) i.e. it converts the int to a byte.

    2. Then the byte 0b11111111 (= 0xFF) is assigned to resigned.

    So it seems nothing really happens to the actual signed value for the unsigned int to be possible.

    But something did happen! The & operator started with the sign-extended value 0b11111111111111111111111111111111 (which represents -1) and converted it to 0b00000000000000000000000011111111 (which represents 255).

    And you can verify that something happened by running this:

      byte signed  = -1; // -—> 0b11111111
  int x = signed;
  System.out.println("x: " + x); // prints -1

    The fact that you ended up with the same value as you started with doesn't mean that nothing happened. Something did happen. You converted the byte value -1 to the int value 255, and then you converted it back to the byte -1.

    You can also verify this by reading the relevant sections of the Java Language Specification. (Let me know if you need the references.)

    Does masking actually do anything to the (byte) value of an integer or it’s just a way to flag how javac should treat the integer?

    Masking really did something.

    If you were to look at the bytecodes produced by the javac compiler you would see codes to do the byte to int conversion, the masking and the assignment. Then in the other direction, you would see a bytecode to do an int to byte conversion.

    However, that is an implementation detail.