How to save 16bit depth Image to file in Arcore (java)

Question

I want to save the depth info from the arcore to the storage. Here is the example from the developer guide.

  public int getMillimetersDepth(Image depthImage, int x, int y) {
  // The depth image has a single plane, which stores depth for each
  // pixel as 16-bit unsigned integers.
  Image.Plane plane = depthImage.getPlanes()[0];
  int byteIndex = x * plane.getPixelStride() + y * plane.getRowStride();
  ByteBuffer buffer = plane.getBuffer().order(ByteOrder.nativeOrder());
  short depthSample = buffer.getShort(byteIndex);
  return depthSample;
}

So I want to save this bytebuffer into a local file, but my output txt file is not readable. How could I fix this? Here is what I have

Image depthImage = frame.acquireDepthImage();
Image.Plane plane = depthImage.getPlanes()[0];
int format = depthImage.getFormat();
ByteBuffer buffer = plane.getBuffer().order(ByteOrder.nativeOrder());
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
File mypath=new File(super.getExternalFilesDir("depDir"),Long.toString(lastPointCloudTimestamp)+".txt");
FileChannel fc = new FileOutputStream(mypath).getChannel();
fc.write(buffer);
fc.close();
depthImage.close();

I tried to decode them with

String s = new String(data, "UTF-8");
System.out.println(StandardCharsets.UTF-8.decode(buffer).toString());

but the output is still strange like this

    .03579:<=>@ABCDFGHJKMNOPRQSUWY]_b

Answer 1

In order to obtain the depth data provided by the ARCore session, you need to write bytes into your local file. A Buffer object is a container, it countains a finite sequence of elements of a specific primitive type (here bytes for a ByteBuffer). So what you need to write inside your file is your data variable that corresponds to the information previously stored in the buffer (according to buffer.get(data)).

It works fine for me, I managed to draw the provided depth map within a python code (but the idea behind the following code can be easily adapted to a java code):

depthData = np.fromfile('depthdata.txt', dtype = np.uint16) 
H = 120 
W = 160
def extractDepth(x):
    depthConfidence = (x >> 13) & 0x7 
    if (depthConfidence > 6): return 0 
    return x & 0x1FFF 
depthMap = np.array([extractDepth(x) for x in depthData]).reshape(H,W)
depthMap = cv.rotate(depthMap, cv.ROTATE_90_CLOCKWISE)

For further details, read the information concerning the format of the depthmap (DEPTH16) here: https://developer.android.com/reference/android/graphics/ImageFormat#DEPTH16 You must also be aware that the depthmap resolution is set to 160x120 pixels and is oriented according to a landscape format.

Make also sure to surround your code by a try/catch code bloc in case of a IOException error.