Look rotation with offset

Question

I’m working on a mechanism to control the position and rotation of an object around an anchorPoint relative to a controlPoint in 3D space. The object can have any initial rotation, necessitating the use of a rotational offset in order to maintain a relative alignment when the controlPoint is moved. Neither the control nor anchor points exist within the unity scene hierarchy as they’re stored as Vector3’s. The object however, utilizes unity’s Transform and may have one or more parent objects each with their own rotation.

The expected behavior is much like that of a local rotation gizmo where you click on the ring of the gizmo to set the rotation of the object.

I've tried without success to implement this using FromToRotation, LookRotation as rotations from identity as well as incremental rotations from the previous rotation state.

The below was the closest i got, however if the object or it's parent have a Z rotation upon calculating the offset, the object then rotates as expected but also includes an undesired roll around the look axis.

public Vector3 anchorPoint = Vector3.zero;
public Vector3 controlPoint = Vector3.left;

Quaternion rotationOffset = Quaternion.identity;

//Called when the object is "attached" to the anchor to store the difference between it's rotation and the directional vector from the control to the anchor
public void StoreOffset() {
    //The unit vector pointing from the controlPoint to it's anchorPoint
    Vector3 controlFwd = (anchorPoint - controlPoint).normalized;

    //Generate a rotation using the direction vector while respecting the objects local up.
    Quaternion lookRot = Quaternion.LookRotation(controlFwd, transform.up);

    //Subtract the object's rotation from the lookRotation to determine the rotational offset between the two.
            //Invert the offset rotation to save on calculations during update
            rotationOffset = Quaternion.Inverse(Quaternion.Inverse(transform.rotation) * lookRot);
}

//Called any time the control point is moved to update the objects rotation
public void UpdateRotation() {
    //The unit vector pointing from the controlPoint to it's anchorPoint
    Vector3 controlFwd = (anchorPoint - controlPoint).normalized;

    //Generate a rotation using the above direction vector while respecting the objects local up.
    Quaternion lookRot = Quaternion.LookRotation(controlFwd, transform.up);

    //Assign the new look rotation to the object then apply the offset rotation.
    //ToLocalRotation is a Quaternion extension that converts rotations from world to local space
            transform.localRotation = (lookRot * rotationOffset).ToLocalRotation(transform);
}

Example Animations

The first .gif shows the correct functionality so long as the parent object has no rotation.

The second .gif shows errant rotation if the parent has a Z rotation of 10°

Answer 1

So before storeoffset happens, you have a current rotation R. You want to calculate an offset X such that from the local axes at rotation R which would produce the look rotation where local forward = control->anchor and local up ≈ the up from R.

That is to say,

R * X = lookrotation(anchor - control, transform.up)

So, in order to find X, multiply both sides of the equation by inverse(R) on the left side:

inverse(R) * R * X = inverse(R) * lookrotation(anchor-control, transform.up)
                 X = inverse(R) * lookrotation(anchor-control, transform.up)

So, the end of StoreOffset should rather be:

rotationOffset = Quaternion.Inverse(transform.rotation) * lookRot;

Then, in UpdateRotation you have X and the lookRot with an up from a ~close-enough~ transform, so solve for R by multiplying both sides by inverse(X) on the right:

R * X              = lookrotation(anchor-control, transform.up)
R * X * inverse(X) = lookrotation(anchor-control, transform.up) * inverse(X)
                 R = lookrotation(anchor-control, transform.up) * inverse(X)

So, the end of UpdateRotation should rather be:

transform.rotation = lookRot * Quaternion.inverse(rotationOffset);

Or, since it's likely that UpdateRotation will be called much more often than StoreOffset, do the inversion before the assignment:

rotationOffsetInverse = Quaternion.inverse(Quaternion.Inverse(transform.rotation) 
                                           * lookRot);
// ...
transform.rotation = lookRot * rotationOffsetInverse;

So, altogether:

public Vector3 anchorPoint = Vector3.zero;
public Vector3 controlPoint = Vector3.left;

Quaternion rotationOffsetInverse = Quaternion.identity;

Quaternion GetControlLookRot()
{
    // The unit vector pointing from the controlPoint to it's anchorPoint
    Vector3 controlFwd = (anchorPoint - controlPoint).normalized;

    // Generate a rotation using the direction vector while respecting the 
    // objects local up.
    return Quaternion.LookRotation(controlFwd, transform.up);
}

// Called when the object is "attached" to the anchor to store the difference between 
// its rotation and the directional vector from the control to the anchor
public void StoreOffset()
{
    rotationOffsetInverse = Quaternion.inverse(Quaternion.Inverse(transform.rotation) 
                                        * GetControlLookRot());
}

// Called any time the control point is moved to update the objects rotation
public void UpdateRotation() 
{
    // Assign the new look rotation to the object then apply the offset rotation.
    transform.rotation = GetControlLookRot() * rotationOffsetInverse;
}