OpenGL How to use a invisible mask to hide objects behind it

Question

I have an OpenGL problem to solve. I have an object/mesh A, an object/mesh B and a background texture C.

Initially the framebuffer is filled with background texture C. We draw both A & B in the framebuffer. We want to keep object A visible, and object B always invisible.

In the beginning, A is in front of B. During rotation, at a certain angle, B is in front of A based on the depth test result, but since B is always invisible, B's part should be filled with background C.

Does anyone know a simple approach to solve this issue?

Is stencil test a good approach? Basically set object B with a color, compare the color of B with background C, and show the background C when the test fail.

Does anyone have any sample code I can read?

Answer 1

The easiest solution is to:

1. draw C;
2. draw B with the colour mask preventing writes to the frame buffer (but don't touch the depth mask, so that writes are still made to the depth buffer);
3. draw A, subject to the depth test.

The specific thing to use is glColorMask — if you supply GL_FALSE for each channel via that then subsequent geometry won't write any colour output. But assuming you haven't touched glDepthMask it'll still write depth output.

So, you've probably currently got the code:

drawBackground(C);
render(A);
render(B);

You'd just adapt that to:

drawBackground(C);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
render(B);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
render(A);