Java OpenCV Aruco Mat

Question

I'm trying to feed an Android camera preview image into OpenCV to detect Aruco codes. Based on previous work, the current flow is as follows:

1. I have a YUV Image.Plane.
2. I feed it into a ZXing PlanarYUVLuminanceSource.
3. On this I call getMatrix(), which gives me a byte[] of luminance values.
4. Examining https://github.com/jsmith613/Aruco-Marker-Tracking-Android/, I notice that it is calling (CvCameraViewFrame).rgba(), and the returned Mat is of type CV_8UC4. I notice that calling (Mat).get(0, 0) yields a double[4] such as {1, 4, 2, 255}, and I infer these correspond to RGBA. This Mat is then passed to (MarkerDetector).detect().
5. I therefore construct a Mat of the same form and type and load it with data.
6. Mat mat = new Mat(width, height, CvType.CV_8UC4, new Scalar(0.0, 0.0, 0.0, 255.0));
7. (for x and y:)
8. int lum = matrix[y * w + x] & 0xFF; mat.put(x, y, new double[]{lum, lum, lum, 255});
9. I then give this Mat to the detector.

It works, but the for loop is slow - it takes about a second to copy all the pixels in. I strongly suspect there's a faster way - surely there's a way to pass a plain byte array into Mat? One that will still work with (MarkerDetector).detect()?

My code, once I have the PlanarYUVLuminanceSource (source), is as follows:

Mat mat = new Mat(width, height, CvType.CV_8UC4, new Scalar(0.0, 0.0, 0.0, 255.0));
byte[] matrix = source.getMatrix();
double[] pixel = new double[]{0,0,0,255};
for (int y = 0; y < height; y++) {
  for (int x = 0; x < width; x++) {
    int luminance = matrix[y * width + x] & 0xFF;
    pixel[0] = luminance;
    pixel[1] = luminance;
    pixel[2] = luminance;
    mat.put(x, y, pixel);
  }
}
Vector<Marker> markers = new Vector<>();
mMarkerDetector.detect(mat, result, mCameraParameters, Constants.ARUCO_MARKER_SIZE, null);

Answer 1

Ok, I got it working:

Mat mat = new Mat(height, width, CvType.CV_8UC4);
byte[] matrix = source.getMatrix();
byte[] temp = new byte[width * height * 4];
int i = 0;
mat.get(0, 0, temp);
for (int y = 0; y < height; y++) {
  for (int x = 0; x < width; x++) {
    byte luminance = matrix[y * width + x];
    temp[i++] = luminance;
    temp[i++] = luminance;
    temp[i++] = luminance;
    temp[i++] = (byte)255;
  }
}
mat.put(0, 0, temp);
Vector<Marker> markers = new Vector<>();
mMarkerDetector.detect(mat, result, mCameraParameters, Constants.ARUCO_MARKER_SIZE, null);

If you alter MarkerDetector to skip Imgproc.cvtColor and use in instead of grey, you can instead do:

Mat mat = new Mat(height, width, CvType.CV_8UC1);
byte[] matrix = source.getMatrix();
byte[] temp = new byte[width * height];
int i = 0;
mat.get(0, 0, temp);
System.arraycopy(matrix, 0, temp, 0, width * height);
mat.put(0, 0, temp);
Vector<Marker> markers = new Vector<>();
mMarkerDetector.detect(mat, result, mCameraParameters, Constants.ARUCO_MARKER_SIZE, null);

Side note - I haven't done enough testing to be sure, but it seems like if you reuse mMarkerDetector, detection gets progressively slower over time, but making a new one each time stays comparatively fast.

Also, note btw that I originally had the dimensions of the matrix swapped - it didn't seem to affect my original code, but it did once I was feeding in raw arrays.

(Note that all this code has been somewhat altered in transit between my IDE and my browser - it SHOULD work, though.)