What's wrong with this simple OpenGL/JOGL stencil test?

Question

I'm learning how to use a stencil buffer, but so far have been unsuccessful at getting a even a simple example to work. In fact, despite trying various combinations of parameters for glStencilOp and glStencilFunc I have not been able to see any evidence that the stencil buffer is working at all. I'm starting to suspect my graphics driver (Mac Pro, Mac OS X 10.8.5) or JOGL (2.0.2) doesn't support it... or I'm missing something really basic.

Here's what I'm seeing:

I'm expecting to see the red diamond clipped by the green diamond. What am I doing wrong?

public class Test {

    public static void main(String[] args) {
        GLProfile glprofile = GLProfile.getDefault();
        final GLCapabilities glcapabilities = new GLCapabilities(glprofile);
        final GLCanvas glcanvas = new GLCanvas(glcapabilities);
        final GLU glu = new GLU();

        glcanvas.addGLEventListener(new GLEventListener() {

            @Override
            public void reshape(GLAutoDrawable glautodrawable, int x, int y, int width, int height) {}



            @Override
            public void init(GLAutoDrawable glautodrawable) {
                GL2 gl = glautodrawable.getGL().getGL2();

                glcapabilities.setStencilBits(8);

                gl.glMatrixMode(GLMatrixFunc.GL_PROJECTION);
                gl.glLoadIdentity();

                glu.gluPerspective(45, 1, 1, 10000);
                glu.gluLookAt(0, 0, 100, 0, 0, 0, 0, 1, 0);

                gl.glMatrixMode(GLMatrixFunc.GL_MODELVIEW);
                gl.glLoadIdentity();
            }



            @Override
            public void dispose(GLAutoDrawable glautodrawable) {}



            @Override
            public void display(GLAutoDrawable glautodrawable) {
                GL2 gl = glautodrawable.getGL().getGL2();

                gl.glEnable(GL.GL_STENCIL_TEST);

                gl.glClearStencil(0x0);
                gl.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT | GL.GL_STENCIL_BUFFER_BIT);

                gl.glStencilFunc(GL.GL_ALWAYS, 1, 1);
                gl.glStencilOp(GL.GL_REPLACE, GL.GL_REPLACE, GL.GL_REPLACE);
            gl.glStencilMask(0xFF);
                //gl.glColorMask(false, false, false, false);
                //gl.glDepthMask(false);

                gl.glColor3f(0, 1, 0);
                gl.glBegin(GL2.GL_QUADS);
                gl.glVertex2f(-25.0f, 0.0f);
                gl.glVertex2f(0.0f, 15.0f);
                gl.glVertex2f(25.0f, 0.0f);
                gl.glVertex2f(0.0f, -15.0f);
                gl.glEnd();

            gl.glStencilMask(0);
                gl.glStencilFunc(GL2.GL_EQUAL, 1, 1);
                gl.glStencilOp(GL2.GL_KEEP, GL2.GL_KEEP, GL2.GL_KEEP);
                //gl.glColorMask(true, true, true, true);
                //gl.glDepthMask(true);

                gl.glColor3f(1, 0, 0);
                gl.glBegin(GL2.GL_QUADS);
                gl.glVertex2f(-20.0f, 0.0f);
                gl.glVertex2f(0.0f, 20.0f);
                gl.glVertex2f(20.0f, 0.0f);
                gl.glVertex2f(0.0f, -20.0f);
                gl.glEnd();
            }
        });

        final JFrame jframe = new JFrame("One Triangle Swing GLCanvas");
        jframe.addWindowListener(new WindowAdapter() {

            @Override
            public void windowClosing(WindowEvent windowevent) {
                jframe.dispose();
                System.exit(0);
            }
        });

        jframe.getContentPane().add(glcanvas, BorderLayout.CENTER);
        jframe.setSize(640, 480);
        jframe.setVisible(true);
    }
}

Answer 1

You need a call to glStencilMask() it's what controls what gets written or not. Set it to do or don't write, draw a stencil (in your case, the diamond), set the glStencilMask() again, and then draw what you want to get clipped.

This has a good sample: Stencil Buffer explanation

EDIT:

OK, I think I found the problem. You need to set your capabilities up at the top of the program.

final GLCapabilities glcapabilities = new GLCapabilities(glprofile);
glcapabilities.setStencilBits(8);
final GLCanvas glcanvas = new GLCanvas(glcapabilities);

The important part being: glcapabilities.setStencilBits(8);

Thanks to: enabling stencil in jogl