I want to generate a ray casted from camera in view space. I do it in this way:
//C++
...
glViewport(wid,hei);
...
//fragment.glsl
vec4 ndc;
ndc.x = gl_FragCoord.x/wid * 2 -1;//map x to range (-1,1)
ndc.y = gl_FragCoord.y/hei * 2 -1;
ndc.z = -1;// because (nearPlane,farPlane) is mapped to (-1,1)
ndc.w = 1;
float w = 1 / gl_FragCoord.w;
vec4 clip = ndc * w;
vec4 view = inverse(project) * clip; // transform from clip space to view space
Ray ray;
ray.origin = vec3(0);
ray.direction = view.xyz;
what I did is first calculating the pixel's position in view space, and then generating a ray from camera position (which is vec3(0) in view space).
I don't know whether this method is correct.
OK, finally, I made it. It should be like this:
We assume that (vx,vy,vz,1) is the coordinate in view space.
(px,py,pz,w) = project * (vx,vy,vz,1)
where (px,py,pz,w) is the coorinate in project space. Another information is that after multiplying with project mat, the w will be -vz. So:
(px,py,pz,w) = (px,py,pz,-vz)
then, we do clip division:
(nx,ny,nz,1) = (px/-vz,py/-vy,pz/-vz,1).
So, from screen coodinate (sx,sy), we can generate a ray in view space in such way:
vec4 NDC ;
NDC.x = gl_FragCoord.x/window_width * 2 - 1;
NDC.y = gl_FragCoord.y/window_height * 2 - 1;
NDC.z = -1; // near plane
NDC.w = 1;
float w = -NearPlane; // the value should be positive, because in view space both NearPlane and FarPlane locates at negative z axis, so zy is minus and -zy is positive.
vec4 clip = NDC * w;
vec3 pos_in_view_space = inverse(project) * clip;
Ray ray;
ray. Origin = vec3(0);
ray. Direction = normalize(pos_in_view_space - ray.origin);
I tried this way and it worked well.