Frustum Culling With View Matrix

Question

In a GLSL shader I need to omit a few tessellation patches to drastically increase performance. These patches are triangles with given world coordinates for each vertex. However, when I convert these coordinates into view space for frustum culling, there is a margin of error.

This is the original terrain.

This is how the error affects it on the top.

This is a closeup of a section with dirt.

These errors happen namly around the top of the screen but also the sides and the bottom. Here is the code I use to determine if I should exclude the triangle (in GLSL).

bool inFrustum( vec3 p,vec3 q,vec3 r) {
    vec4 Pclip = camera * vec4(p, 1.0f);
    vec4 Qclip = camera * vec4(q, 1.0f);
    vec4 Rclip = camera * vec4(r, 1.0f);
    if(((-Pclip.w>Pclip.x&&-Qclip.w>Qclip.x&&-Rclip.w>Rclip.x)||    (Pclip.x>Pclip.w&&Qclip.x>Qclip.w&&Rclip.x>Rclip.w))||
       ((-Pclip.w>Pclip.y&&-Qclip.w>Qclip.y&&-Rclip.w>Rclip.y)||(Pclip.y>Pclip.w&&Qclip.y>Qclip.w&&Rclip.y>Rclip.w))||
       ((-Pclip.w>Pclip.z&&-Qclip.w>Qclip.z&&-Rclip.w>Rclip.z)||(Pclip.z>Pclip.w&&Qclip.z>Qclip.w&&Rclip.z>Rclip.w))){
    return false;
    }
    else{
    return true;
    }
}

I would greatly appreciate any help given! Behemyth

Answer 1

In my shader I use the following to cull patches:

bool visible(vec3 vert)
{
    int clipoffset = 5; //a bit offset because of displacements
    vec4 p = MVP*vec4(vert,1);
    return !(( p1.x < -(p1.w+clipoffset))||
            ( p.x >  (p.w+clipoffset))||
            ( p.y < -(p.w+clipoffset))||
            ( p.y >  (p.w+clipoffset))||
            ( p.z < -(p.w+clipoffset))||
            ( p.z >  (p.w+clipoffset)));
}

and it looks like this from above:

PS: I use quads tessellation so I check if one of the vertices is in frustum:

if( visible(inPos[0])||
            visible(inPos[1])||
            visible(inPos[2])||
            visible(inPos[3]))
        {
            outt[0] = calcTessellationLevel(inPos[3],inPos[0]);
            outt[1] = calcTessellationLevel(inPos[0],inPos[1]);
            outt[2] = calcTessellationLevel(inPos[1],inPos[2]);
            outt[3] = calcTessellationLevel(inPos[2],inPos[3]);

            inn[1] = (outt[0]+outt[2])/2;
            inn[0] = (outt[1]+outt[3])/2;
        }

EDIT: In your code maybe the (and) || operators caused the problem, try that without brackets after every second statement:

if(S1||S2||S3||S4) 

instead of

if((S1||S2)||(S3||S4))

EDIT:: hmmm....I haven't looked at the date it was asked, dont know how I've found it....O.o