Search code examples
mediapipe

How does faceeffect decide where to attach the 3dmesh? e.g attach the axis to the nose

Looking into below Mediapipe's faceeefect module's graph definition

  node_options: {
[type.googleapis.com/mediapipe.SwitchContainerOptions] {
  contained_node: {
    calculator: "FaceGeometryEffectRendererCalculator"
    node_options: {
      [type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
        effect_texture_path: "mediapipe/graphs/face_effect/data/axis.pngblob"
        effect_mesh_3d_path: "mediapipe/graphs/face_effect/data/axis.binarypb"
      }
    }
  }

I further checked the calculator code of FaceGeometryEffectRendererCalculator, I couldn't pinpoint the code where it determines the exact location the renderer render the axis. Maybe I don't understand the OpenGL well which leads to the misunderstanding.

Can someone help shed some lights where should I look for?

Thanks！

Solution

  • By digging into the document and paper I get to understand that the estimated face geometry is aligned with the canonical face model. You can refer the pager https://arxiv.org/pdf/1907.06724.pdf and the explanation here