We all have used augmented face texture either on Instagram or Snapchat or any other application. I'm trying to develop a sample app where one can try on some ar objects on their faces. So far, I've been able to display that object on the face.
Then I'm displaying a list of items below ( on the screen ) so that user can switch the object to some other object ( similar to trying out new filters on instagram / snapchat ).
This is the default code for one object:
Texture.builder()
.setSource(this, R.drawable.fox_face_mesh_texture)
.build()
.thenAccept(texture -> faceMeshTexture1 = texture);
ArSceneView sceneView = arFragment.getArSceneView();
sceneView.setCameraStreamRenderPriority(Renderable.RENDER_PRIORITY_FIRST);
Scene scene = sceneView.getScene();
scene.addOnUpdateListener(new Scene.OnUpdateListener() {
@Override
public void onUpdate(FrameTime frameTime) {
if (faceMeshTexture1 == null)
return;
Collection<AugmentedFace> faceList =
sceneView.getSession().getAllTrackables(AugmentedFace.class);
for (AugmentedFace face : faceList) {
if (!faceNodeMap.containsKey(face)) {
AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
faceNode.setParent(scene);
faceNode.setFaceMeshTexture(faceMeshTexture1);
faceNodeMap.put(face, faceNode);
}
}
Iterator<Map.Entry<AugmentedFace, AugmentedFaceNode>> iter =
faceNodeMap.entrySet().iterator();
while (iter.hasNext()) {
Map.Entry<AugmentedFace, AugmentedFaceNode> entry = iter.next();
AugmentedFace face = entry.getKey();
if (face.getTrackingState() == TrackingState.STOPPED) {
AugmentedFaceNode faceNode = entry.getValue();
faceNode.setParent(null);
iter.remove();
}
}
}
});
What I've tried
In button onClick - I created a new Texture by doing Texture.builder().setSource ...
1) I tried to just write the for loop for (AugmentedFace face : faceList) { ... } But it did nothing.
2) I tried to include the whole contents of onUpdate() function. But that didn't work too.
3) Then I tried creating 2 textures and using a boolean variable to switch between those textures ( in the onUpdate method ) by reversing the boolean value ( on Button click ). It looks something like this:
Texture.builder()
.setSource(this, R.drawable.fox_face_mesh_texture)
.build()
.thenAccept(texture -> faceMeshTexture1 = texture);
Texture.builder()
.setSource(this, R.drawable.red_lipstick)
.build()
.thenAccept(texture -> faceMeshTexture2 = texture);
ArSceneView sceneView = arFragment.getArSceneView();
sceneView.setCameraStreamRenderPriority(Renderable.RENDER_PRIORITY_FIRST);
Scene scene = sceneView.getScene();
scene.addOnUpdateListener(new Scene.OnUpdateListener() {
@Override
public void onUpdate(FrameTime frameTime) {
if (faceMeshTexture1 == null || faceMeshTexture2 == null)
return;
Collection<AugmentedFace> faceList =
sceneView.getSession().getAllTrackables(AugmentedFace.class);
for (AugmentedFace face : faceList) {
if (!faceNodeMap.containsKey(face)) {
AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
faceNode.setParent(scene);
if( !redOrFox ) {
faceNode.setFaceMeshTexture(faceMeshTexture1);
Log.d(TAG, "onUpdate: redorfox:" + redOrFox);
}
else {
faceNode.setFaceMeshTexture(faceMeshTexture2);
Log.d(TAG, "onUpdate: else redorfox:" + redOrFox);
}
faceNodeMap.put(face, faceNode);
}
}
Iterator<Map.Entry<AugmentedFace, AugmentedFaceNode>> iter = faceNodeMap.entrySet().iterator();
while (iter.hasNext()) {
Map.Entry<AugmentedFace, AugmentedFaceNode> entry = iter.next();
AugmentedFace face = entry.getKey();
if (face.getTrackingState() == TrackingState.STOPPED) {
AugmentedFaceNode faceNode = entry.getValue();
faceNode.setParent(null);
iter.remove();
}
}
}
});
mBtnBlueLip.setOnClickListener(v -> {
redOrFox = true;
Log.d(TAG, "onCreate: blue clicked");
});
mBtnRedLip.setOnClickListener(v -> {
redOrFox = false;
Log.d(TAG, "onCreate: red clicked");
});
But that didn't do anything too. At this point I'm clueless about what to do. There are barely any documentation regarding augmented faces and a few sample apps ( but in Kotlin ). A similar question was found ( for 3D ) but with no answers.
edit: Also after debugging, it seems like the onUpdate function is called only once. I don't understand what is going on here?
I found the solution in this kotlin app developed by Kristina Simakova.
It seems like I was on the right track. What I missed was that the face object was already in the faceList array so it wasn't at all going in this if condition:
if (!faceNodeMap.containsKey(face)) { ... }
and since I had no else, nothing happened.
The solution would be have another boolean variable which checks whether the button was clicked or not!
Boolean isFox = false, changeModel = false;
...
scene.addOnUpdateListener(frameTime -> {
...
for (AugmentedFace face : faceList) {
if (!faceNodeMap.containsKey(face)) {
AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
faceNode.setParent(scene);
faceNode.setFaceMeshTexture(faceMeshTexture1);
faceNodeMap.put(face, faceNode);
}
else if (changeModel) { // check whether we want to change texture
if(isFox)
faceNodeMap.get(face).setFaceMeshTexture(faceMeshTexture1);
else
faceNodeMap.get(face).setFaceMeshTexture(faceMeshTexture2);
}
}
changeModel = false; // after the for loop ends
...
mBtnBlueLip.setOnClickListener(v -> {
isFox = true;
changeModel = true;
});
mBtnRedLip.setOnClickListener(v -> {
changeModel = true;
isFox = false;
});
Kristina used a similar approach. Since I was testing the app and had only 2 textures, I used a boolean variable isFox. You can also maintain an array of your textures and keep only one faceMeshTexture variable which gets the value according to which button was clicked.