I'm using the dot product of these two vectors but its not working as intended

Question

I am using the dot product of the players position and the cursors position and then using the cosign of that value to rotate the player appropriately, but is glitching and acting very strangly, switching from negative to positive numbers seemingly randomly, and i cant seem to fix it.

camRotation = Mathf.Cos(player.transform.rotation.x + player.transform.rotation.y * Input.mousePosition.x + Input.mousePosition.y);
player.transform.rotation = Quaternion.Lerp(Quaternion.Euler(0, 0, player.transform.rotation.z), Quaternion.Euler(0, 0, camRotation * 100), 1 * Time.deltaTime);

Answer 1

From what I can see from your question, you are trying to spin the player based on the current mouse position. However, there are a couple of issues with your approach:

1. You should use Quaternion.Slerp rather than Quaternion.Lerp, as lerp simply interpolates each field and is almost never what you are looking for. Quaternion.Slerp moves as if spinning a sphere
2. Unnecessary cosine calculation - Quaternion.Euler takes an angle, so just using the mouse position as the angle should acheive the desired effect
3. Spinning on the wrong axis - you probably meant to use the y-axis rather than the z-axis

This is probably what you are looking for:

camRotation = Input.mousePosition.x;
player.transform.rotation = Quaternion.Slerp(Quaternion.Euler(0, player.transform.rotation.y, 0), Quaternion.Euler(0, camRotation * 5, 0), 1 * Time.deltaTime);

Please comment if I have misinterpreted :)