Constrain pitch, yaw, & roll

Question

I've a rotation represented as a quaternion and am trying to constrain the pitch, yaw, & roll axes. I tried doing so thusly:

public struct Orientation
{
    public Vector3 up, forward;

    public Orientation(Vector3 up, Vector3 forward)
    {
        this.up = up;
        this.forward = forward;
    }
}

public static Orientation[] orientations = new Orientation[3]
{
    new Orientation(Vector3.right, Vector3.up),
    new Orientation(Vector3.up, Vector3.forward),
    new Orientation(Vector3.forward, Vector3.right)
};

public enum Axis
{
    Pitch,
    Yaw,
    Roll
};

private Vector3 ConstrainAxis(Vector3 vector, Axis axis, float from, float to)
{
    Orientation orientation = orientations[(int)axis];

    float theta = (to - from) * 0.5F;

    Vector3 cons = Quaternion.AngleAxis(from + theta, orientation.up) * orientation.forward;
    Vector3 proj = Vector3.ProjectOnPlane(vector, orientation.up);

    return ConstrainVector(cons.normalized, proj.normalized, theta);
}

private Vector3 ConstrainVector(Vector3 from, Vector3 to, float angle)
{
    float theta = Mathf.Abs(angle / Vector3.Angle(from, to));

    if(theta < 1.0F)
    {
        return Vector3.Slerp(from, to, theta);
    }

    return to;
}

Which turned out to be nothing more than an over-complicated way of constraining the individual components of an euler angle representation, of which both are subject to a strange jittering issue (gimbal lock related?).

What is the best approach to constraining these axes?

Answer 1

For joint constraints it is common practice to use "swing twist" parametrization. To represent current rotation as "swing twist" for quaternions, theare are good decomposition https://web.archive.org/web/20160909191250/https://www.alinenormoyle.com/weblog/?p=726

And constraint for "swing" and "twist" can be done with quaternions.

if we want to constrain swing to +-30 degrees , pseudocode looks like

Quaternion swing;
const double maxMagnitude = sin(0.5 * toRad(30));
const double maxMagnitudeW = sqrt(1.0 - maxMagnitude * maxMagnitude);
if (swing.vec().normSqr() > maxMagnitude * maxMagnitude)
{
    swing.vec() = swing.vec().normalized() * maxMagnitude;
    swing.w() = maxMagnitudeW;
}