Find ViewPort Rectangle in 3D world Space (Opengl)

Question

I need help about the matrix Transformation to find the corners of my viewPort in 3d coordinates(World Space).

I have done some test but i cannot find the solution.

Step 1: I have Projection and Model Matrix avaible (ViewPort size too). To find the center of my "Screen" i have used this: OpenGL.UnProject(0.0, 0.0, 0.0) <- this function tell me where is the center of my screen in 3D space (Correct!) Another approach is to multiply the Coordinate (0, 0, 0) * ProjectionMtx.Inverse.

Step 2: Now i need for example the left top corner of my viewport, how can i find the 3D point in the world space? Probably i should work with the viewport size but how?

This is my unproject method:

        double[] mview = new double[16];
        GetDouble(GL_MODELVIEW_MATRIX, mview);
        double[] prj = new double[16];
        GetDouble(GL_PROJECTION_MATRIX, prj);
        int[] vp = new int[4];
        GetInteger(GL_VIEWPORT, vp);
        double[] r = new double[3];
        gluUnProject(winx, winy, winz, mview, prj, vp, ref r[0], ref r[1], ref r[2]);

For Example: if i have my camera in (-40,0,0) and my vieport [0,0,1258,513] and i unproject my near plane points i have this result:

left_bottom_near =>X=-39.7499881839701,Y=-0.0219584744091603,Z=0.946276352352364 right_bottom_near =>X=-39.7499881839701,Y=-0.0219584744091603,Z=0.946446903614738
left_top_near =>X=-39.7499881839701,Y=-0.0217879231516134,Z=0.946276352352364
right_top_near =>X=-39.7499881839701,Y=-0.0217879231516134,Z=0.946446903614738

I can understand the X value of my points that is ~ to my x world value of my camera position but, what about the Y & Z? I cannot understand.

Answer 1

gluUnProject converts from window coordinates to to world space (or model space).
The view matrix converts from world space to view space.
The projection matrix transforms from view space to normalized device space. The normalized device space is a cube with the left, bottom, near of (-1, -1, -1) and the right, top, far of (1, 1, 1).
Finally the xy coordinates in normalized device space are mapped to the viewport (window coordinates). The viewport is defined by glViewport. The z component is maped to the glDepthRange (defalt [0.0, 1.0]. gluUnProject does the opposite of all this steps.

At orthographic projection the viewing volume is a cuboid. At Perspective projection the viewing volume is a frustum. The projection matrix transforms from view space to normalized device space.
What you can see on the viewport is the projection of the normalized device space to its xy plane.

In general the corners of the viewing volume can be get by un-projecting the 8 corner points of the window coordinates. For that you have to know the viewport rectangle and the depth range. I assume that the viwport has the size of the window (0, 0, widht, height) and that the depth range is [0.0, 1.0]:

left   = 0.0;
right  = width;
bottom = 0.0;
top    = height; 

left_bottom_near  = gluUnProject(left,   bottom, 0.0)
right_bottom_near = gluUnProject(right,  bottom, 0.0)
left_top_near     = gluUnProject(left,   top,    0.0)
right_top_near    = gluUnProject(right,  top,    0.0)
left_bottom_far   = gluUnProject(left,   bottom, 1.0)
right_bottom_far  = gluUnProject(right,  bottom, 1.0)
left_top_far      = gluUnProject(left,   top,    1.0)
right_top_far     = gluUnProject(right,  top,    1.0)