Search code examples
androidc++carcoreglm-math

How to create Anchor in front of camera in ARCore NDK API

I'm trying to create anchor in front of camera using ARCore C API. I extracted current pose of camera, and the documentation says that

-Z pointing in the direction the camera is looking

So I translated matrix of pose to -0.3 by Z axis.

      ArPose *pose = nullptr;
      ArPose_create(mArSession, nullptr, &pose);
      ArCamera_getPose(mArSession, arCamera, pose);

      float poseMatrix[16];
      ArPose_getMatrix(mArSession, pose, poseMatrix);

      glm::mat4 matrix = glm::make_mat4(poseMatrix);
      matrix = glm::translate(matrix, glm::vec3(0, 0, -0.3f));

      ArPose* newPose;
      ArPose_create(mArSession, glm::value_ptr(matrix), &newPose);

      ArAnchor *anchor;
      ArSession_acquireNewAnchor(mArSession, newPose, &anchor);

      ArPose_destroy(pose);
      ArPose_destroy(newPose);

Anchor is placed, but it's placed relative the initial position of device (this also causes that all anchors placed in the same place), not current position, while I'd like to place it in front of camera.

I saw another question with one answer. But I don't understand how to get driftx, so I leaving it to zero. This code:

      ArAnchor *anchor;

      float raw_pose[7] = {0, 0, 0, 1.0, /*THERE WAS driftx -->*/0, 0, -5.0};
      ArPose *arPose;
      ArPose_create(mArSession, raw_pose, &arPose);
      ArSession_acquireNewAnchor(mArSession, arPose, &anchor);

still placing anchor relative the initial position:

If I rotate the camera and run this code again anchor will be placed in the same place. I saw another method in Java. It takes transition and rotation as arguments (may be identical to example above). However I'm using NDK API and I also not sure that's it's going to work in Java.

Full repo with code to reproduce: https://github.com/vladd11/ar-shop Code will run after tap to screen.

UPD: did some research. As I see, I need to this:

  1. Get camera pose (ArCamera_getPose) and convert it to matrix;
  2. In nutshell, I need to find a point where user is looking currently using GLM. enter image description here
  3. Then I need to pass this matrix to ArPose to create Anchor. But I don't see the way to do it. So I need to work with raw pose, which includes rotation quartineon and transform and write all the Math myself, but I don't understand what I should do.
Solution

  • First, you need to get view matrix of AR Camera and convert it to GLM matrix.

    float rawMatrix[16];
ArCamera_getViewMatrix(mArSession, arCamera, rawMatrix);
glm::mat4 matrix = glm::make_mat4(rawMatrix);

    View matrix is commonly used to convert world coordinates to camera position (to apply perspective and camera transformation).

    The thing that I like to do now is to convert relative camera position (glm::vec4(1.f, 1.f, -5.f, 0.f)) to world position.

    glm::vec4 newPosition = glm::vec4(1.f, 1.f, -5.f, 0.f) * matrix;

    This position may be applied to new AR pose, then to new AR anchor.

    float newRawPose[7] = {
  // Rotation
  0.f, // X
  0.f, // Y
  0.f, // Z
  1.f, // W
  // Position
  newPosition[0], // X
  newPosition[1], // Y
  newPosition[2] //  Z
};

ArPose *newPose;
ArPose_create(mArSession, newRawPose, &newPose);

ArAnchor *anchor;
ArSession_acquireNewAnchor(mArSession, newPose, &anchor);

    Don't forget to destroy new AR pose to avoid memory leaks:

    ArPose_destroy(newPose);

    Full code.