How to achieve downsampling AA/filtering in OpenGL

Question

I'm working on a game that uses pixelart and a camera that will not match the size of the pixel art one to one. To keep the pixels looking like pixels, I want to render the entire game at a higher resolution then downsample it to the actual window resolution (similar to what the GeDoSaTo mod does in Dark Souls 2) and simply use nearest filtering on the in-game textures as mag filter. How can I do this downsampling in code?

Answer 1

I have written some pseudo-code for you that shows how to setup an FBO with dimensions:

  scale * (res_x x res_y).

        ^{2x supersampling would use a scale of 2.0.}

I did some extra work to get you a texture image attachment for your color buffer, so that you can draw this using a textured quad (better performance) instead of blitting. However, since doing that would involve writing a (simple) shader, glBlitFramebuffer (...) was the quickest solution.

Code to Initialize your FBO:

GLuint supersample_fbo,
       supersample_tex,
       supersample_rbo_depth;

glGenTextures (1, &supersample_tex);
glBindTexture (GL_TEXTURE_2D, supersample_tex);

glGenRenderbuffers (1, &supersample_rbo_depth);
glBindRenderbuffer (GL_RENDERBUFFER, supersample_rbo_depth);

// Allocate storage for your texture (scale X <res_x,res_y>)
glTexImage2D  (GL_TEXTURE_2D, 0, GL_RGBA8, res_x * scale, res_y * scale, 0, GL_RGBA, GL_FLOAT, NULL);

// Allocate storage for your depth buffer
glRenderBufferStorage (GL_RENDERBUFFER, GL_DEPTH_COMPONENT24, res_x * scale, res_y * scale);

glGenFramebuffers (1, &supersample_fbo);
glBindFramebuffer (GL_FRAMEBUFFER, supersample_fbo);

// Attach your texture to the FBO: Color Attachment 0.
glFramebufferTexture2D (GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, supersample_tex, 0);

// Attach the depth buffer to the FBO.
glFramebufferRenderbuffer (GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, supersample_rbo_depth);

Code to Draw into your FBO:

glBindFramebuffer (GL_FRAMEBUFFER, supersample_fbo);

// You need to modify the viewport mapping to reflect the difference in size.
// Your projection matrix can stay the same since everything is uniformly scaled.
//
glViewport        (0, 0, res_x * scale, res_y * scale);

  // DRAW

Code to Blit the FBO to the Default Framebuffer:

glBindFramebuffer (GL_READ_FRAMEBUFFER, supersample_fbo); // READ:  Supersampled
glBindFramebuffer (GL_DRAW_FRAMEBUFFER, 0);               // WRITE: Default

// Downsample the supersampled FBO using LINEAR interpolation
glBlitFramebuffer (0,0,res_x * scale, res_y * scale,
                   0,0,res_x,         res_y,
                   GL_COLOR_BUFFER_BIT,
                   GL_LINEAR);

Code to resume drawing into the Default Framebuffer:

// You probably want all subsequent drawing to go into the default framebuffer...
glBindFramebuffer (GL_FRAMEBUFFER, 0);
glViewport        (0,0,res_x,res_y);

There is a lot of missing error checking in this code, and FBOs are very error-prone, but it should get you pointed in the right direction.