Check if a fragment is within a specified area

Question

Consider the following scene.

I transformed pMin and pMax from world space to viewport space. The area bound by pMin and pMax follows the users mouse sliding over the plane (larger rectangle).

Is there a way inside the fragment shader to decide if the fragment lies within the inner area or not? I tried comparing with gl_FragCoord.x and gl_FragCoord.z but it does not yield correct results.

if((gl_FragCoord.x < splitMax.x && gl_FragCoord.x > splitMin.x) 
&& (gl_FragCoord.z < splitMax.z && gl_FragCoord.z > splitMin.z)){
    //within area following the mouse
} else {
   //outside of area following the mouse
}

In cascaded shadow mapping, shadow maps are chosen based on the fragment's z value and whether it lies inside the computed frustum split z values. I'm trying to do the same only that I want my look up to also consider the x coordinate.

Answer 1

Thanks to a guy in ##opengl on freenode, I managed to get this working the following way:

vertex shader: Transform the incoming vertex to world space

out vec4 worldPos;
...
worldPos = modelMatrix * vec4(vertex, 1.0);

fragment shader: Send in pMin and pMax in world space coordinates

in vec3 pMin, pMax;
in vec4 worldPos;
...
if((worldPos.x > pMin.x && worldPos.x < pMax.x) && (worldPos.z > pMin.z && worldPos.z < pMax.z)){
   FragColor = vec4(1.0, 0.0, 0.0, 1.0);
} else {
   FragColor is scene lighting
}

Result: