Hbase acts as a source and sink for my Map reduce job. I have written my custom writable class called (vector writable) and it has two fields in it.
private DoubleVector vector; // It is a Double Array
private byte[] rowKey; // The row key of the Hbase
My mapper emits this as its value, hence I have implemented write and read methods in my vectorWritable class
@Override
public final void write(DataOutput out) throws IOException {
writeVectorCluster(this.vector, this.rowKey, out);
}
@Override
public final void readFields(DataInput in) throws IOException {
this.vector = readVector(in);
this.rowKey = readRowKey(in);
}
public static void writeVectorCluster(DoubleVector vector, byte[] rowkey, DataOutput out)
throws IOException {
out.writeInt(vector.getLength());
for (int i = 0; i < vector.getDimension(); i++) {
out.writeDouble(vector.get(i));
}
int length = rowkey.length;
out.writeInt(length);
//Is this the right way ?
out.write(rowkey);
}
public static DoubleVector readVector(DataInput in) throws IOException {
int length = in.readInt();
DoubleVector vector = null;
vector = new DenseDoubleVector(length);
for (int i = 0; i < length; i++) {
vector.set(i, in.readDouble());
}
return vector;
}
@SuppressWarnings("null")
public static byte[] readRowKey(DataInput in) throws IOException {
int length = in.readInt();
byte [] test = null;
for (int i = 0; i < length; i++) {
// getting null pointer exception here
test[i] = in.readByte();
}
return test;
}
I get a NullPointerException when I try to read the rowKey from the input stream. The readVector method works fine though and I am getting the right value.
How can I write the byte array in the DataInput Stream so that I could retrive it in my Output streams
UPDATE : SOLVED This is the update of my rowKey method which works fine. Thanks @Perception
public static byte[] readRowKey(DataInput in) throws IOException {
int length = in.readInt();
byte[] theBytes = new byte[length];
in.readFully(theBytes);
return theBytes;
}
You didn't initialize your byte array:
byte [] test = null;
Should be:
byte [] test = new byte[length];