GLSL unexpected step behaviour

Question

I have an outline shader I adapted from Here, although I only can implement section 1 (depth) because I don't have a normal buffer (which I don't really know what that is). It works pretty good, although with faraway objects they fade out at certain angles:

I would think this is an issue with depth buffer precision, but I believe I am using a 32 bit depth buffer:

glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT32, windowWidth, windowHeight, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_BYTE, NULL); //reserve mem
. . .
glTexImage2DMultisample(GL_TEXTURE_2D_MULTISAMPLE, sampleLevel, GL_DEPTH_COMPONENT32, windowWidth, windowHeight, GL_FALSE); //..

The weird part comes when I try to add a distance cutoff. The original code looks like this:

float depth0 = float(texture(depthTex, vec2(texCoords.x - scale/texSize.x, ((1.f - texCoords.y) - scale/texSize.y))).r); //BL
(repeated 4 times)

And it gives the above result. To add the distance cutoff I fed the depth buffer result into a step func to try and stop outlines rendering too far away (this was because I didn't want the issue of not rendering at certain angles to be as noticeable). The stepped code looks like this:

float depth0 = step(lineCutoff, texture(depthTex, vec2(texCoords.x - scale/texSize.x, ((1.f - texCoords.y) - scale/texSize.y))).r); //BL

If I set the lineCutoff to 0.0001f or 10.f I get no render, but if I set it to 1.f i get the following result:

This is weird for two reasons:

1. Far away objects are rendered nicely (top left box)
2. Outlines are conjoined (boxes on the right)

My question is: why is this happening, and is there a way to get far away objects to render properly with out conjoining outlines.

NOTE: the two rounded rectangles are close enough to eachother to not draw an outline between them unless I am close, also they shouldn't even be writing to the depth buffer (they are transparent) so I am not sure why they get any outline. I also tried just wrapping the sampler calls in a float to see if that was it but no visual change occured.

The buffer creation first creates a multisampled depth buffer on a framebuffer which is drawn to, this is copied to a regular framebuffer, which is drawn on a quad on which this shader operates.

Most of the shader code:

#version 330 core
uniform sampler2D depthTex;

const float scale = 1.f;  //must be integers
const float lineCutoff = 1.f; //lower = further

void main()
{
    vec2 texSize = textureSize(depthTex, 0); //depthBuffer dimensions
    float depth0 = step(lineCutoff, texture(depthTex, vec2(texCoords.x - scale/texSize.x, ((1.f - texCoords.y) - scale/texSize.y))).r); //BL
    float depth1 = step(lineCutoff, texture(depthTex, vec2(texCoords.x + scale/texSize.x, ((1.f - texCoords.y) + scale/texSize.y))).r); //TR
    float depth2 = step(lineCutoff, texture(depthTex, vec2(texCoords.x + scale/texSize.x, ((1.f - texCoords.y) - scale/texSize.y))).r); //BR
    float depth3 = step(lineCutoff, texture(depthTex, vec2(texCoords.x - scale/texSize.x, ((1.f - texCoords.y) + scale/texSize.y))).r); //TL

    float depthEdge = sqrt(pow(depth1 - depth0, 2.f) + pow(depth3 - depth2, 2.f)) * 100.f; //"roberts cross" * intensity
    fragColour = vec4(vec3(step(1.5f, depthEdge)), 1.0f); //output stepped to give clean outlines
}

Answer 1

I fixed it! The planes' conjoined outline was infact a depth buffer precision issue, increasing my near plane fixed them.

The issue was from the step function basically turned the depth buffer into like a binary result instead of a float, if there was any value in the buffer other than 1 (dist = inf) it was black:

I had a feeling the issue was something like this where step was doing not what I thought. To fix it I used this instead of step:

    depth0 = depth0 == 1.f ? 0 : depth0;
    (repeat for each one)

Which gives the following result for depth0:

This clearly has more contrast between far objects and infinity which I believe was the underlying issue. If someone has a way to fix this without using what is basically an if statement, then I would still greatly appreciate (for efficiency sake).

Final result:

NOTE: formatting might be messed up again because stack overflow sometimes deleted my new lines and indents near code blocks and image embeds. I also changed the near plane to 0.25f and far plane to 50.f when calculating my projection matrix (gives more prevision for those planes near the camera). Now to apply anti-aliasing (perhaps FXAA) to smooth out those lines!