Edge/outline detection from texture in fragment shader

Question

I am trying to display sharp contours from a texture in WebGL. I pass a texture to my fragment shaders then I use local derivatives to display the contours/outline, however, it is not smooth as I would expect it to.

Just printing the texture without processing works as expected:

vec2 texc = vec2(((vProjectedCoords.x / vProjectedCoords.w) + 1.0 ) / 2.0,
            ((vProjectedCoords.y / vProjectedCoords.w) + 1.0 ) / 2.0 );
vec4 color = texture2D(uTextureFilled, texc);
gl_FragColor = color;

With local derivatives, it misses some edges:

vec2 texc = vec2(((vProjectedCoords.x / vProjectedCoords.w) + 1.0 ) / 2.0,
            ((vProjectedCoords.y / vProjectedCoords.w) + 1.0 ) / 2.0 );
vec4 color = texture2D(uTextureFilled, texc);
float maxColor = length(color.rgb);
gl_FragColor.r = abs(dFdx(maxColor));
gl_FragColor.g = abs(dFdy(maxColor));
gl_FragColor.a = 1.;

Answer 1

In theory, your code is right.

But in practice most GPUs are computing derivatives on blocks of 2x2 pixels. So for all 4 pixels of such block the dFdX and dFdY values will be the same. (detailed explanation here)

This will cause some kind of aliasing and you will miss some pixels for the contour of the shape randomly (this happens when the transition from black to the shape color occurs at the border of a 2x2 block).

To fix this, and get the real per pixel derivative, you can instead compute it yourself, this would look like this :

// get tex coordinates
vec2 texc = vec2(((vProjectedCoords.x / vProjectedCoords.w) + 1.0 ) / 2.0,
                 ((vProjectedCoords.y / vProjectedCoords.w) + 1.0 ) / 2.0 );

// compute the U & V step needed to read neighbor pixels
// for that you need to pass the texture dimensions to the shader, 
// so let's say those are texWidth and texHeight
float step_u = 1.0 / texWidth;
float step_v = 1.0 / texHeight;

// read current pixel
vec4 centerPixel = texture2D(uTextureFilled, texc);

// read nearest right pixel & nearest bottom pixel
vec4 rightPixel  = texture2D(uTextureFilled, texc + vec2(step_u, 0.0));
vec4 bottomPixel = texture2D(uTextureFilled, texc + vec2(0.0, step_v));

// now manually compute the derivatives
float _dFdX = length(rightPixel - centerPixel) / step_u;
float _dFdY = length(bottomPixel - centerPixel) / step_v;

// display
gl_FragColor.r = _dFdX;
gl_FragColor.g = _dFdY;
gl_FragColor.a = 1.;

A few important things :

• texture should not use mipmaps
• texture min & mag filtering should be set to GL_NEAREST
• texture clamp mode should be set to clamp (not repeat)

And here is a ShaderToy sample, demonstrating this :