POLYGON_SMOOTH in OpenGL doing wierd things

Question

I am trying to get some anti aliasing in my voxel engine, but when I enable the GL_POLYGON_SMOOTH state, I get these weird waves on the border of the polygons

Without:

  

With:

  

Do you have an idea from where this could be from?

Answer 1

That is normal.

You need 3 things for polygon smoothing to work properly:

1. Draw your voxels from back-to-front when you use GL_POLYGON_SMOOTH.
2. A framebuffer that stores destination alpha (e.g. RGBA not RGB)
3. The blend function needs to be GL_SRC_ALPHA_SATURATE, GL_ONE

If you account for all three of those things, the black and white subpixel artifacts will go away.

Since drawing the voxels in a specific order is rather tedious and GL_POLYGON_SMOOTH is deprecated, most people simply use Multisample Anti-Aliasing instead. I would actually suggest you also go that route.