Search code examples
javascriptcanvasspark-ar-studio

2D face tracking on Spark AR

I've been trying to make the face texture in a 2D canvas/plane move only along the X/Y axis, following the movements of the face without rotating, with the 2D background camera texture reflected accurately on top. Right now, when I connect the canvas to the face tracker I'm getting distorted scale, and the 2D plane rotates in 3D space. See below for the current canvas/camera texture/face tracker set-up. Manual scaling results in poor tracking.

Here is my code:

const FaceTracking = require('FaceTracking')
const Scene = require('Scene')

export const Diagnostics = require('Diagnostics');

// Locate the plane in the Scene
// Enable async/await in JS [part 1]
(async function () {
  const [plane] = await Promise.all([
    Scene.root.findFirst('blur_plane')
  ])

  // Store a reference to a detected face
  const face = FaceTracking.face(0)

  // To access scene objects
  const planeTransform = plane.transform
  const faceTransform = face.cameraTransform
  // const blurCanvas = Scene.root.find('canvas0');

  // To access class properties
  planeTransform.rotationX = faceTransform[0]
  planeTransform.rotationY = faceTransform[0]
  planeTransform.rotationZ = faceTransform[0]
})()

This is the current look: current canvas/camera texture and face tracker set-up I want the 375x667px canvas to look exactly like the camera layer beneath it, so that without adjustments to the camera texture the layer would not be visible.

Solution

  • Turns out Facebook has an example that deals with 2D movement but not scale: https://sparkar.facebook.com/ar-studio/learn/reference/classes/facetrackingmodule