Search code examples
opengl-esglslopengl-es-2.0glsles

Why is there a light area between edges when rendering an inner shadow by calculating the nearest distance to an edge?


I'm rendering an inner shadow (same idea as an inner glow, but dark instead of light) by calculating the nearest distance to an edge for each pixel. This is done in the fragment shader by interpolating the vertex position from the vertex shader (using varying), passing the shape's edge points via a uniform array, and calculating the nearest distance by running a point-to-line-segment distance calculation against each edge.

Here's the fragment shader:

precision mediump float;
uniform mediump sampler2D uBaseSampler;
uniform mediump vec2 uVerts[11];
varying mediump vec2 vTexCoord;
varying mediump vec4 vPosition;
float distanceSq(mediump vec2 a, mediump vec2 b)
{
    mediump vec2 c = b - a;
    return dot(c, c);
}
float distSqFromLineSegment(mediump vec2 p, mediump vec2 a, mediump vec2 b)
{
    float lenSq = distanceSq(a, b);
    /*if (lenSq == 0.0) return distanceSq(p, a);*/
    float t = dot(p - a, b - a) / lenSq;
    if (t < 0.0) return distanceSq(p, a);
    else if (t > 1.0) return distanceSq(p, b);
    mediump vec2 proj = a + t * (b - a);
    return distanceSq(p, proj);
}
void main()
{
    float nearestDistSq = 1e3;
    for (lowp int i = 0; i < 10; ++i)
    {
        lowp int j = i + 1;
        float distSq = distSqFromLineSegment(vPosition.xy, uVerts[i], uVerts[j]);
        nearestDistSq = min(nearestDistSq, sqrt(distSq));
    }
    float nearestDist = nearestDistSq;
    float coef = nearestDist;
    lowp vec4 frag = texture2D(uBaseSampler, vTexCoord);
    gl_FragColor = vec4(frag.xyz * coef, 1.0);
}

Here's how it looks when calculating the inner shadow based on edge 1 only:

Inner shadow from edge 1

Here's how it looks when calculating the inner shadow based on edge 2 only:

Inner shadow from edge 2

Notice how the inner shadow is correct for both edges individually.

Now, here's how it looks when calculating the inner shadow based on both edges:

Inner shadow from edges 1 and 2

Notice the inner lightness between the edges? And, for reference, here is a similar outcome when calculating the inner shadow based on all of the edges:

Inner shadow from all edges

When looking at the inner shadows generated by edge 1 and edge 2 independently, it doesn't look like that inner lightness would form by factoring in both edges at the same time. What could be causing it? The points along the lightness should not have a larger distance than their neighboring points because I'm calculating the minimum distance from all of the edges.

Any thoughts?

Update:

Based on some early answers, it looks like my issue is that I've implemented a "darken" algorithm, and combining two gradients this way causes the side effect in my screenshots. For example, if you take my first two screenshots, overlay them in an image editor, and set the second layer to the "darken" blend mode, you'll get my third screenshot.

What I'm really looking for is some kind of multiplication. If I use the "multiply" blend mode in my image editor, I get the desired look:

Inner shadow from edges 1 and 2 multiplied

How would I achieve this? It looks like I have to calculate the inner shadow independently for each edge, then multiply all of the coefficients together, instead of calculating one coefficient based on the closest edge.

Update 2:

Implementing a multiplicative blend mode results in darker areas around the vertices of the shape. Turned out worse than my original version. I've decided to stick with the original version as it's fast, correct, and the undesired side effects are minimized by scaling the coefficient around 0.8-1.0 instead of 0.0-1.0. I've also accepted the answer that led me down this path.


Solution

  • There's nothing wrong with the picture... OpenGL is drawing exactly what you asked for, which is a brightness related to the distance from an edge. The distance is not going to be a smooth function, so your eye sees edges where the derivative of the distance function suddenly changes.

    You can test this yourself. Open up an image editing program such as Gimp or Photoshop or whatever, and make two linear gradients. For the second gradient, choose "darken" as the composition mode for the gradient. I got the following image:

    Gradient image

    You can see that there is a similar 45 degree line here.

    If you want to get rid of the sharp edges, you'll have to do something more, like blur the result. Distance fields in general are going to have sharp edges.