I'm using the ARToolKit for Android to build an AR app. I can apply the Projection Matrix and the Marker Transformation Matrix in OpenGL without problem, as explained in the ARSimple example. However, I did not find a way to apply correctly these to the jPCT-AE camera. Here is what I did for the camera:
Camera cam = world.getCamera();
Matrix projMatrix = new Matrix();
projMatrix.transformToGL();
projMatrix.setDump(ARToolKit.getInstance().getProjectionMatrix());
cam.setPosition(projMatrix.getTranslation());
cam.setBack(projMatrix);
and for the object:
Matrix objMat = new Matrix();
objMat.transformToGL();
objMat.setDump(ARToolKit.getInstance().queryMarkerTransformation(markerID));
cube.setTranslationMatrix(objMat);
cube.setRotationMatrix(objMat);
It almost works: I can see the 3D object if the marker is placed at the center of the screen. However when I move the marker it quickly disappears off screen. Also, the cube (and other models I tried to load) seems to render in some sort of "inverted" way. For what I read on the web the ARToolKit matrices are relative to OpenGL world coordinates (while jPCT-AE has its own coordinates), and also that the projection matrix of jPCT-AE is built internally based on the fov, near and far clipping plane, position and rotation, and then I cannot set it directly.
How do I translate the projection matrix and marker matrix to the jPCT-AE engine?
Reviewing my code, it seems JPCT-AE does not get the position and back vector correctly from the matrix (although I see no reason why it does not), but it does when you split them in separate vectors. This are just my findings from trial and error.
This is how I did it for the camera, using the direction and up vectors.
float[] projection = ARToolKit.getInstance().getProjectionMatrix();
Matrix projMatrix = new Matrix();
projMatrix.setDump(projection);
projMatrix.transformToGL();
SimpleVector translation = projMatrix.getTranslation();
SimpleVector dir = projMatrix.getZAxis();
SimpleVector up = projMatrix.getYAxis();
mCamera.setPosition(translation);
mCamera.setOrientation(dir, up);
And then for the model I extract translation and rotation. It is important to clear the translation, since it is not an absolute position, but a modification to the current position. I think this may be your main problem why the objects move out of the screen.
float[] transformation = ARToolKit.getInstance().queryMarkerTransformation(markerID);
Matrix dump = new Matrix();
dump.setDump(transformation);
dump.transformToGL();
mModel.clearTranslation();
mModel.translate(dump.getTranslation());
mModel.setRotationMatrix(dump);
Also, you should do the transformToGl after calling setDump, I think that is the reason why you see them inverted.
Finally, you should reuse the matrix between executions instead of creating a new object every frame, for optimization.