Search code examples
c++glslraytracingvulkancompute-shader

Incorrect ray direction from inverse vp matrix and camera position

I have a problem with my ray generation that I do not understand. The direction for my ray is computed wrongly. I ported this code from DirectX 11 to Vulkan, where it works fine, so I was surprised I could not get it to work:

vec4 farPos = inverseViewProj * vec4(screenPos, 1, 1);
farPos /= farPos.w;

r.Origin = camPos.xyz;
r.Direction = normalize(farPos.xyz - camPos.xyz);

Yet this code works perfectly:

vec4 nearPos = inverseViewProj * vec4(screenPos, 0, 1);
nearPos /= nearPos.w;
vec4 farPos = inverseViewProj * vec4(screenPos, 1, 1);
farPos /= farPos.w;

r.Origin = camPos.xyz;
r.Direction = normalize(farPos.xyz – nearPos.xyz);

[Edit] Matrix and camera positions are set like this:

const glm::mat4 clip(1.0f, 0.0f, 0.0f, 0.0f, 0.0f, -1.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.5f, 0.0f, 0.0f, 0.0f, 0.5f, 1.0f);
projMatrix = clip * glm::perspectiveFov(FieldOfView, float(ViewWidth), float(ViewHeight), NearZ, FarZ);
viewMatrix = glm::inverse(glm::translate(glm::toMat4(Rotation), -Position));

buffer.inverseViewProjMatrix = glm::inverse(projMatrix * viewMatrix);
buffer.camPos = viewMatrix[3];

[Edit2] What I see on screen is correct if I start at the origin. However, if I move left, for example, it looks as if I am moving right. All my rays seem to be perturbed. In some cases, strafing the camera looks as if I am moving around a different point in space. I assume the camera position is not equal to the singularity of my perspective matrix, yet I can not figure out why.

I think I am misunderstanding something basic. What am I missing?

Solution

  • Thanks to the comments I have found the problem. I was building my view matrix incorrectly, in the exact same way as in this post:

    glm::inverse(glm::translate(glm::toMat4(Rotation), -Position));

    This is equal to translating first and then rotating, which of course leads to something unwanted. In addition, the Position was negative and camPos was obtained using the last column of the view matrix instead of the inverse view matrix, which is wrong.

    It was not noticable with my fractal raycaster simply because I never moved far away from the origin. That, and the fact that there is no point of reference in such an environment.