I am writing the entire buffer pointed to by TangoImageBuffer.data to a file.
// open a file
std::fstream f(app.getNextFileName(), std::ios::out | std::ios::binary);
// write the entire stride * height * 4 buffer into the file
f.write((const char *) tango_image_buffer->data,
tango_image_buffer->stride * buffer->height * 4 * sizeof(uint8_t)
);
f.close();
I then export the file to my PC and visualize it using python-opencv and numpy:
import sys
import cv2
import numpy as np
# dimensions of the image
stride = 1280
width = 1280
height = 720
# I am using 3 channels so that the resulting image is not
# transluscent
channels = 3
input_filename = sys.argv[1]
output_filename = sys.argv[2]
# load the image buffer into a list
data = np.fromfile(input_filename, dtype=np.uint8)
# create a height x width x channels matrix with the datatype uint8
# and all elements set to zero
img = np.zeros((height, width, channels), dtype=np.uint8);
# map elements in array to image matrix
for y in range(0, height):
for x in range(0, width):
img[y, x, 0] = data[y * stride + x + 2] #blue
img[y, x, 1] = data[y * stride + x + 1] #green
img[y, x, 2] = data[y * stride + x + 0] #red
# display and save the resulting image
cv2.namedWindow("tango-rgba")
cv2.imshow("tango-rgba", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.imwrite(output_filename, img)
Unfortunately, the resulting image looks gray
The image only appears to be gray. Upon closer inspection of the pixels, you will find that the red, green, and blue components of the pixels are different.
EDIT: I have changed this post to specify that the buffer I am using is in RGBA8888 format. From the Project Tango Reference, "Only RGBA 8888 is provided"
EDIT: It actually seems RGBA8888 format is not used. rhashimoto suggests that the format is YV12.
EDIT: The image format is actually NV21 (See answer below).
It seems that as of writing this post, the Project Tango Reference really is incorrect.
The image format is NV21.
char format_str[100] = {0};
switch (tango_image_buffer->format)
{
case TANGO_HAL_PIXEL_FORMAT_RGBA_8888:
sprintf(format_str, "%s", "RGBA8888");
break;
case TANGO_HAL_PIXEL_FORMAT_YV12:
sprintf(format_str, "%s", "YV12");
break;
case TANGO_HAL_PIXEL_FORMAT_YCrCb_420_SP:
sprintf(format_str, "%s", "NV21");
break;
default:
break;
}
__android_log_print(ANDROID_LOG_VERBOSE, "Capture Info",
"stride: %u, width: %u, height: %u, timestamp: %lf, frame: %llu, format: %s",
tango_image_buffer->stride,
tango_image_buffer->width,
tango_image_buffer->height,
tango_image_buffer->timestamp,
tango_image_buffer->frame_number,
format_str);
The output of the above piece of code is:
V/Capture Info﹕ stride: 1280, width: 1280, height: 720, timestamp: 22157.368703, frame: 0, format: NV21
Here is my new visualization code in python using opencv and numpy:
import sys
import cv2
import numpy as np
input_filename = sys.argv[1]
output_filename = sys.argv[2]
# dimensions of the image
stride = 1280
width = 1280
height = 720
channels = 4
# load file into buffer
data = np.fromfile(input_filename, dtype=np.uint8)
# create yuv image
yuv = np.ndarray((height + height / 2, width), dtype=np.uint8, buffer=data)
# create a height x width x channels matrix with the datatype uint8 for rgb image
img = np.zeros((height, width, channels), dtype=np.uint8);
# convert yuv image to rgb image
cv2.cvtColor(yuv, cv2.COLOR_YUV2BGRA_NV21, img, channels)
# display and save the resulting image
cv2.namedWindow("tango-rgba")
cv2.imshow("tango-rgba", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
cv2.imwrite(output_filename, img)
and the resulting image.
TLDR: The image format is NV21. OpenCV provides functionality for converting NV21 to RGB. The result looks great.