Search code examples
androidopengl-es3drajawali

Labeling parts of 3D model with Android and OpenGL ES/Rajawali

I have imported a model (e.g. a teapot) using Rajawali into my scene. What I would like is to label parts of the model (e.g. the lid, body, foot, handle and the spout) using plain Android views, but I have no idea how this could be achieved. Specifically, positioning the labels on the right place seems challenging. The idea is that when I transform my model's position in the scene, the tips of the labels are still correctly positioned

Rajawali tutorial show how Android views can be placed on top of the scene here https://github.com/Rajawali/Rajawali/wiki/Tutorial-08-Adding-User-Interface-Elements . I also understand how using the transformation matrices a 3D coordinate on the model can be transformed into a 2D coordinate on the screen, but I have no idea how to determine the exact 3D coordinates on the model itself. The model is exported to OBJ format using Blender, so I assume there is some clever way of determining the coordinates in Blender and exporting them to a separate file or include them somehow in the OBJ file (but not render those points, only include them as metadata), but I have no idea how I could do that.

Any ideas are very appreciated! :)

Solution

  • I would use a screenquad, not a view. This is a general GL solution, and will also work with iOS.

    You must determine the indices of the desired model vertices. Using the text rendering algo below, you can just fiddle them until you hit the right ones.

    1. Create a reasonable ARGB bitmap with same aspect ratio as the screen.
    2. Create the screenquad texture using this bitmap
    3. Create a canvas using this bitmap
    4. The rest happens in onDrawFrame(). Clear the canvas using clear paint.
    5. Use the MVP matrix to convert desired model vertices to canvas coordinates.
    6. Draw your desired text at the canvas coordinates
    7. Update the texture.
    • Your text will render very precisely at the vertices you specfied. The GL thread will double-buffer and loop you back to #4. Super smooth 3D text animation!
    • Use double floating point math to avoid loss of precision during coordinate conversion, which results in wobbly text. You could even use the z value of the vertex to scale the text. Fancy!

    • The performance bottleneck is #7 since the entire bitmap must be copied to GL texture memory, every frame. Try to keep the bitmap as small as possible, maintaining aspect ratio. Maybe let the user toggle the labels.

    • Note that the copy to GL texture memory is redundant since in OpenGL-ES, GL memory is just regular memory. For compatibility reasons, a redundant chunk of regular memory is reserved to artificially enforce the copy.