Search code examples
javakotlinjvmbinaryfilesjava-io

Read binary stream containing unsigned numbers

I want to read binary file containing 32-bit unsigned integers and 8-bit unsigned integers. I already know DataInputStream but its method readInt returns signed integers and there is no method for reading unsigned ints (there are such methods for 16-bit and 8-bit integers).

Reading separate bytes and concatenating them bitwise is the “official” way to do it? Would reading bytes into ByteArray and composing integers from them using bitshifts and bitwise ors significantly decrease performance?

Solution

  • You can use

    long value = Integer.toUnsignedLong​(dataInputStream.readInt());

    This is equivalent to the pre-Java 8 code

    long value = dataInputStream.readInt() & 0xFFFFFFFFL;

    The key point is that signed or unsigned are just different interpretations of the bit pattern, but to read the four byte quantity, readInt() is always sufficient. The operation above converts to a signed long, a datatype capable of covering all values of unsigned int.

    But since the int does already hold all information, there is no need to convert it to a long immediately. The Two’s Complement used to represent the signed numbers even allows performing basic operations, i.e. +, -, and *, without differentiating between signed and unsigned numbers. For other operations, Java 8 introduced methods to perform them by interpreting the int value as unsigned:

    A practical example, I encountered, is parsing class files. These files have sized encoded as unsigned int at some place, but with most standard Java APIs, class files are delivered as byte array or ByteBuffer instances, which can not contain more than 2³¹ bytes. So dealing with larger numbers is an unnecessary complication for something that can’t be correct anyway, as a class file containing such a large size specification must be truncated.

    So the code to handle this looks basically like:

    int size = input.readInt();
if(Integer.compareUnsigned(size, Integer.MAX_VALUE)>0) throw new IllegalArgumentException(
    "truncated class file (attribute size "+Integer.toUnsignedString(size)+')');
// just use the int value

    or without Java 8 features
    (even simpler, as long as the reader understands the Two’s Complement):

    int size = input.readInt();
if(size < 0) throw new IllegalArgumentException(
    "truncated class file (attribute size "+(size&0xFFFFFFFFL)+')');
// just use the int value

    (see also this answer)