Can't get stencil buffers to work in OpenTK, simple 2D quads

Question

I've been trying with no success to get stencil testing to work in my OpenTK 2D game - I want to only draw parts of a texture below values of say 1 in a stencil buffer. Have spent ages reading up on stencils and how they work but can't find a single example in c#. The code below is my attempt at porting the code from the link at the bottom of:

opengl stencil buffer - not quite got it working

Which is: http://pastebin.com/vJFumvQz

But I get no masking at all, just a green rectangle inside a blue. I suspect I need to initialize the buffers in some way? Any help would be greatly appreciated.

public class StencilTest : GameWindow
{
    int stencilBuffer;

    public StencilTest() :
        base(1, 1, GraphicsMode.Default, "Stencil test")
    {
        Width = 1500;
        Height = 800;
        VSync = VSyncMode.On;
        GL.ClearColor(Color4.Red);
        ClientSize = new Size(1500, 800);
        this.Location = new System.Drawing.Point(100,300);
        GL.Viewport(0, 0, Width, Height);


        GL.GenRenderbuffers(1, out stencilBuffer);
        GL.RenderbufferStorage(RenderbufferTarget.Renderbuffer, RenderbufferStorage.Depth24Stencil8, 1500, 800);
        GL.BindRenderbuffer(RenderbufferTarget.Renderbuffer, stencilBuffer);
    }

    protected override void OnRenderFrame(FrameEventArgs e)
    {
        GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.StencilBufferBit);

        GL.MatrixMode(MatrixMode.Projection);
        GL.LoadIdentity();
        GL.Ortho(-10, 10, -10, 10, -1, 1);

        GL.MatrixMode(MatrixMode.Modelview);
        GL.LoadIdentity();

        GL.Disable(EnableCap.StencilTest);
        GL.StencilMask(0x0);

        draw_background();

        GL.Enable(EnableCap.StencilTest);
        GL.StencilMask(0x1);
        GL.StencilFunc(StencilFunction.Always, 0x1, 0x1);
        GL.StencilOp(StencilOp.Keep, StencilOp.Keep, StencilOp.Replace);
        GL.ColorMask(false, false, false, false);
        GL.DepthMask(false);

        draw_triangle();

        GL.StencilMask(0x1);
        GL.StencilFunc(StencilFunction.Notequal, 0x1, 0x1);
        GL.StencilOp(StencilOp.Keep, StencilOp.Keep, StencilOp.Keep);
        GL.ColorMask(true, true, true, true);
        GL.DepthMask(true);

        draw_blue_square();

        SwapBuffers();
        base.OnRenderFrame(e);
    }


    void draw_background()
    {
        //big blue square
        GL.Color4(Color4.Blue);
        GL.Begin(PrimitiveType.Quads);
        GL.Vertex3(5, 5, 0);
        GL.Vertex3(-5, 5, 0);
        GL.Vertex3(-5, -5, 0);
        GL.Vertex3(5, -5, 0);
        GL.End();
    }

    void draw_triangle()
    {
        /* transparent triangle */
        GL.Color3(Color.White);
        GL.Begin(PrimitiveType.Triangles);
        GL.Vertex3(-4, -4, 0);
        GL.Vertex3(4, -4, 0);
        GL.Vertex3(0, 4, 0);
        GL.End();
    }

    void draw_blue_square()
    {
        /* blue square */
        GL.Color4(Color4.Green);
        GL.Begin(PrimitiveType.Quads);
        GL.Vertex3(3, 3, 0);
        GL.Vertex3(-3, 3, 0);
        GL.Vertex3(-3, -3, 0);
        GL.Vertex3(3, -3, 0);
        GL.End();
    }
}

Answer 1

It looks like you don't have a stencil buffer. This part of your code is an attempt to get a stencil buffer:

GL.GenRenderbuffers(1, out stencilBuffer);
GL.RenderbufferStorage(RenderbufferTarget.Renderbuffer, RenderbufferStorage.Depth24Stencil8, 1500, 800);
GL.BindRenderbuffer(RenderbufferTarget.Renderbuffer, stencilBuffer);

but it's not what you need in this case. This allocates a stencil buffer that you could use when rendering to a Frame Buffer Object (aka FBO) by attaching it as the depth-stencil attachment of the FBO. But in your code, you're rendering to the default framebuffer, so the above will not have any effect.

When rendering to the default framebuffer, you typically have to request all the buffers you need during initial setup. I'm not familiar with OpenTK, but based on what I found in the GraphicsContext documentation, you need to create a GraphicsMode instance that looks something like this:

new GraphicsMode(32, 24, 8, 0)

and pass it along when setting up your context. This specifies that you want a 32-bit color buffer, a 24-bit depth buffer, and a 8-bit stencil buffer. I suspect that this should replace the GraphicsMode.Default you are using in your code.