Basically I have a local space where y points down and x points right, like
|
|
---------> +x
|
| +y
and the center is at (320, 240), so the upper left corner is (0,0). Some windowing system uses it.
So I have this code
auto const proj = glm::ortho(0.0f, 640.0f, 0.0f, 480.0f);
auto const view = glm::lookAt(
glm::vec3(320.0f, 240.0f, -1.0f), //eye position
glm::vec3(320.0f, 240.0f, 0.0f), //center
glm::vec3(0.0f, -1.0f, 0.0f) //up
);
I imagine I need to look at the (320,240,0) from the negative position of z towards positive z, and the "up" direction is negative y.
However it doesn't seems to provide the right result
auto const v = glm::vec2(320.0f, 240.0f);
auto const v2 = glm::vec2(0.0f, 0.0f);
auto const v3 = glm::vec2(640.0f, 480.0f);
auto const result = proj * view * glm::vec4(v, 0.0f, 1.0f); //expects: (0,0), gives (-1,-1)
auto const result2 = proj * view * glm::vec4(v2, 0.0f, 1.0f); //expects: (-1,1), gives (-2,0)
auto const result3 = proj * view * glm::vec4(v3, 0.0f, 1.0f); //expects: (1,-1), gives (0,-2)
That's not how view and projection work. The view matrix tells you which point should be mapped to [0,0] in view space. It seems you try to map the center of the visible area to [0,0], but then you use a projection matrix which assumes that [0,0] is the top-left corner.
Since you first apply the view matrix, that gives a result of view * glm::vec4(v, 0.0f, 1.0f) = [0,0]. Then the projection matrix get's applied where you defined that 0,0 as the top-left corner, thus proj * [0,0] will result in [-1,-1].
I'm not 100% sure what you want to achieve, but if you want to use the given projection matrix, then the view matrix has to transform the scene in a way that the top-left point of the visible area get's mapped to [0,0].
You can also adjust the project to use the range [-320, 320] (and respectively [-240, 240]) and keep mapping the center to [0,0] with the view matrix.