How to clip with circles in OpenGL

I'm wondering if it is possible to simulate the effect of looking through the keyhole in OpenGL.

I have my 3D scene drawn but I want to make everythig black everything except a central circle.

I tried this solution but its doing the completely opposite of what I want:

// here i draw my 3D scene 

// Begin 2D orthographic mode
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glLoadIdentity();

GLint viewport [4];
glGetIntegerv(GL_VIEWPORT, viewport);

gluOrtho2D(0, viewport[2], viewport[3], 0);

glMatrixMode(GL_MODELVIEW);
glPushMatrix();
glLoadIdentity();

// Here I draw a circle in the center of the screen
float radius=50;
glBegin(GL_TRIANGLE_FAN);
glVertex2f(x, y);
for( int n = 0; n <= 100; ++n )
{
    float const t = 2*M_PI*(float)n/(float)100;
    glVertex2f(x + sin(t)*r, y + cos(t)*r);
}
glEnd();

// end orthographic 2D mode
glMatrixMode(GL_PROJECTION);
glPopMatrix();
glMatrixMode(GL_MODELVIEW);
glPopMatrix();

What I get is a circle drawn in the center, but I would like to obtain its complementary...

Solution

Like everything else in OpenGL, there are a few ways to do this. Here are two off the top of my head.

Use a circle texture: (recommended)

Draw the scene.
Switch to an orthographic projection, and draw a quad over the entire screen using a texture which has a white circle at the center. Use the appropriate blending function:
```
glEnable(GL_BLEND);
glBlendFunc(GL_ZERO, GL_SRC_COLOR);
/* Draw a full-screen quad with a white circle at the center */
```

Alternatively, you can use a pixel shader to generate the circular shape.

Use a stencil test: (not recommended, but it may be easier if you don't have textures or shaders)

Clear the stencil buffer, and draw the circle into it.

glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, 1, 1);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
/* draw circle */

Enable the stencil test for the remainder of the scene.

glEnable(GL_STENCIL_TEST)
glStencilFunc(GL_EQUAL, 1, 1);
glStencileOp(GL_KEEP, GL_KEEP, GL_KEEP);
/* Draw the scene */

Footnote: I recommend avoiding use of immediate mode at any point in your code, and using arrays instead. This will improve the compatibility, maintainability, readibility, and performance of your code --- a win in all areas.