Rotating around the wrong axis

Question

I'm trying to rotate a camera around a point in space, something like this:

Where the red dot is typically going to be the center of something like an object.

The user can rotate the camera with their mouse, where moving vertically should rotate vertically (x axis) and horizontally should rotate horizontally (y axis).

What I want

Rotations to the y axis are always to the apparent y axis of the camera, and same with x. For instance if the user moves the mouse up, it should always look like you are moving up and over the object the camera is focusing on. If the user moves the mouse from left to right, it should look like the object is spinning on its y axis (or that the camera is moving around its y axis).

So if the user moves the mouse a bit to the left, and then starts moving it up, it should still look like they are going over the top of the object from the new perspective, rather than rotating at an angle along the world's actual x axis.

This seems standard in most 3D software I've used, for example.

What I've tried

With the help of a lot of questions here, I've gotten pretty close to what I want, but not quite.

For the following bits of code: view_ is the 4x4 view matrix, origin_ would be the spot in the picture we're rotating around, position_ is the blue line in the image representing how far back the camera is from the origin (or any camera pan, but that isn't implemented yet), and pitch, yaw, and roll are the amounts we want to rotate (in this case, roll is always zero).

My best result so far was with this:

// rotation_ is a vec3 here
rotation_ += glm::vec3(glm::radians(pitch), glm::radians(yaw), glm::radians(roll) );
glm::mat4 rot = glm::rotate( glm::mat4(1.0f), rotation_.x, glm::vec3(1.0f, 0.0f, 0.0f) );
rot = glm::rotate( rot, rotation_.y, glm::vec3(0.0f, 1.0f, 0.0f) );
rot = glm::rotate( rot, rotation_.z, glm::vec3(0.0f, 0.0f, 1.0f) );
view_ = glm::translate(glm::translate( glm::mat4(1.0f), position_ ) * rot,  origin_);

Close, but rotating by ~90 degrees along the x axis causes further rotations around the y axis appear to be around the z axis. I can't seem to replicate this with the y axis to x axis though, so I'm curious if this is actually gimbal lock or something else.

To solve this, I tried storing rotation_ as a quaternion, and getting the rotation like this:

// rotation_ is a quaternion here.
rotation_ *= glm::fquat(glm::vec3(glm::radians(pitch), glm::radians(yaw), glm::radians(roll)));
glm::vec3(1.0f, 0.0f, 0.0f) );
view_ = glm::translate(glm::translate( glm::mat4(1.0f), position_ ) * rot,  origin_);

This is actually worse. It looks like I'm always rotating along the world's axes rather than by the camera's orientation, which isn't what I want.

How can I get the camera to always look like it rotates vertically when moving the mouse up/down, and always look like it rotates horizontally when moving left/right?

Answer 1

I was combining my quaternions in the wrong order!

// rotation_ is a quaternion here.
rotation_ *= glm::fquat(glm::vec3(glm::radians(pitch), glm::radians(yaw), glm::radians(roll)));
glm::vec3(1.0f, 0.0f, 0.0f) );
view_ = glm::translate(glm::translate( glm::mat4(1.0f), position_ ) * rot,  origin_);

The line

rotation_ *= glm::fquat(glm::vec3(glm::radians(pitch), glm::radians(yaw), glm::radians(roll)));

Should instead have been

rotation_ = glm::fquat(glm::vec3(glm::radians(pitch), glm::radians(yaw), glm::radians(roll))) * rotation_;

Like this the quaternion solution works like I expected it to.

If I understand correctly, this is because I wanted to first rotate to the new delta rotation, and then add the old rotation to it. This lets me apply the new rotation to the camera's axes first before taking the rest of the rotation into account.

Doing it the other way around would have it rotate to the old rotation, and then add the new rotation to that, which results in the new rotation not being applied in the directions I was expecting.