From mouse coordinates to world coordinates through glUnproject

Question

I am trying to retrieve my mouse coordinates (in a QGLWidget through Qt) to estimate its current 2D coordinates in the virtual world (all my vertices have z=0).

To do so, I wrote this:

modelViewMatrix = np.asarray(matView*matModel)
        viewport = glGetIntegerv(GL_VIEWPORT)

        z = 0

        x, y, z = GLU.gluUnProject(float(self.mouseX), float(self.mouseY), float(z), model = modelViewMatrix,
                             proj = np.asarray(matProj), view = viewport)

The matModel is always the identity matrix (numpy.eye(4)) while the matView and matProj matrices are computed thanks to the lookat(eye, target, up) and perspective(fovy, aspect, near, far) methods. The matView matrix is the only one which change through mouse events in my app.

I also provide my vertex shader:

VERTEX_SHADER = """
#version 440 core

uniform float scale;
uniform mat4 Model;
uniform mat4 View;
uniform mat4 Projection;

in vec2 position;   
in vec4 color;     

out vec4 v_color;


void main()
{
    gl_Position =  Projection*View*Model*vec4(scale*position, 0.0, 1.0);
    v_color = color;

}
 """

To test my code snippet, I draw a square with vertices (-1,-1), (-1,1), (1,-1) and (1,1). But when I move my mouse to a corner I do not get any +-1 coordinates.

So, I guess there is something wrong in my code...

Answer 1

I finally solved it using this code:

depth = glReadPixels(self.mouseX, self.height()-self.mouseY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT)
z = 2*depth -1

mouseWorld = (self.viewport*self.matProj*self.matView*self.matModel).I*np.matrix([[self.mouseX, self.mouseY, z, 1]]).T