Finding perpendicular vector to a given vector and in respect to former position in 3D space

I have 3 vectors, xyz, and i want to change only 1 and then calculate the other 2 in respect to their former directions (the closest possible to them). And i want to do it the most efficient way in terms of memory usage/game design.

small illustration of the problem

EDIT (after some constructive comments from Stef):

What we have:

x,y,z 3d vector, normalized and with same origin (0,0,0)
y' (the up vector changes depending on the surface)

What (i believe) we need:

z' or x' (and all mutually perpendicular)

What i tried:

z' with the help of the y - y' angle around the x-axis and vice versa with x' and the z-axis. i think this "could" work with the proper linear algebra knowledge but would still mean an euler rotation, right? i prefer to only rotate with quaternions even though i still earn to figure them out
using the cross-product of y' and x to get z' and then creating the crossproduct of z' and y' to get x'. in theory it should work? my 3 dot-product are never 0, not one of them

What i desire:

changing the x and z vector based on the angle of y and y' via a quaternion. i am working in godot and still figuring things out.. that said, i just found this:

Quaternion Quaternion ( Vector3 arc_from, Vector3 arc_to )

Constructs a quaternion representing the shortest arc between two points on the surface of a sphere with a radius of 1.0.

Solution

If I understand correctly, you want a quaternion that represents the shortest rotation needed to transform a vector into another, so you can apply it your custom basis.

We can start by figuring out the rotation axis for this rotation transformation...

I'll be writing GDScript. And I'll be assuming the vectors are non-zero and finite.

var axis := arc_from.cross(arc_to)

Except that does not work when the vectors are in the same or opposite direction.

If you check the dot product and it says the vectors are in the same direction, then you need no rotation at all. But if the dot product says they are in opposite directions, you need a half turn rotation, yet you need to decide over which axis.

Here is one possible solution:

var axis := arc_from.cross(arc_to)
if axis.is_zero_approx():
    var longest_axis := arc_to.max_axis_index()
    var with := Vector3.ONE
    with[longest_axis] = 0.0
    axis = arc_to.cross(with)

Here I've created a new Vector3 that will be 0.0 in the axis that arc_to has longest magnitud and 1.0 in the others. So it is guaranteed to not be in the same direction, and then I'm using it do the cross product with arc_to, which should give a vector perpendicular to arc_to.

This is a way to get a vector perpendicular to an arbitrary vector in 3D space. I remind you that problem has infinite solutions. And this is just one approach.

Of course, we should normalize the axis to make sure it is of length 1.0:

var axis := arc_from.cross(arc_to)
if axis.is_zero_approx():
    var longest_axis := arc_to.max_axis_index()
    var with := Vector3.ONE
    with[longest_axis] = 0.0
    axis = arc_to.cross(with)

axis = axis.normalized()

Now we can figure out the angle for our rotation:

var axis := arc_from.cross(arc_to)
if axis.is_zero_approx():
    var longest_axis := arc_to.max_axis_index()
    var with := Vector3.ONE
    with[longest_axis] = 0.0
    axis = arc_to.cross(with)

axis = axis.normalized()
var angle := arc_from.signed_angle_to(arc_to, axis)

And make a quaternion for it:

var axis := arc_from.cross(arc_to)
if axis.is_zero_approx():
    var longest_axis := arc_to.max_axis_index()
    var with := Vector3.ONE
    with[longest_axis] = 0.0
    axis = arc_to.cross(with)

axis = axis.normalized()
var angle := arc_from.signed_angle_to(arc_to, axis)
return Quaternion(axis, angle)

I believe this is sufficient for your needs.

Addendum

is_zero_approx

This line is checking if the axis vector is approxiately a zero vector:

if axis.is_zero_approx():

We only care about the case when it is a zero vector, so ideally we could check against zero. But since axis is the result of floating point operations we do not have a guarantee that it will be exactly zero.

The check is equivalent to this:

if absf(axis.x) < 0.00001 and absf(axis.y) < 0.00001 and absf(axis.z) < 0.00001:

max_axis_index

Next, this line give you an index 0, 1, or 2 depending which axis is longer x, y, or z:

var longest_axis := arc_to.max_axis_index()

We can do the same thing like this:

var longest_axis := 2
if arc_to.x >= arc_to.y and arc_to.x >= arc_to.z:
    longest_axis = 0
elif arc_to.y >= arc_to.z:
    longest_axis = 1

The array access to the Vector3 allows us to get or set the values by index. So the following will result in a vector with only that axis set to 0.0 and the others to 1.0:

var with := Vector3.ONE
with[longest_axis] = 0.0

So we can do this another way:

var with := Vector3(1.0, 1.0, 1.0)
if arc_to.x >= arc_to.y and arc_to.x >= arc_to.z:
    with.x = 0.0
elif arc_to.y >= arc_to.z:
    with.y = 0.0
else:
    with.z = 0.0

Or if you prefer:

var with := Vector3(1.0, 1.0, 0.0)
if arc_to.x >= arc_to.y and arc_to.x >= arc_to.z:
    with = Vector3(0.0, 1.0, 1.0)
elif arc_to.y >= arc_to.z:
    with = Vector3(1.0, 0.0, 1.0)