How to convert relative GPS coordinates to a "local custom" x, y, z coordinate?

Question

Let's say I know two persons are standing at GPS location A and B. A is looking at B.

I would like to know B's (x, y, z) coordinates based on A, where the +y axis is the direction to B (since A is looking at B), +z is the vertically to the sky. (therefore +x is right-hand side of A)

I know how to convert a GPS coordinate to UTM, but in this case, a coordinate system rotation and translation seem needed. I am going to come up with a calculation, but before that, will there be some codes to look at?

I think this must be handled by many applications, but I could not find so far.

Answer 1

1. Convert booth points to 3D Cartesian

   GPS suggest WGS84 so see How to convert a spherical velocity coordinates into cartesian

2. Construct transform matrix with your desired axises

   see Understanding 4x4 homogenous transform matrices. So you need 3 perpendicular unit vectors. The Y is view direction so

   Y = normalize(B-A);
   

   one of the axises will be most likely up vector so you can use approximation

   Z = normalize(A);
   

   and as origin you can use point A directly. Now just exploit cross product to create X perpendicular to both and make also Y perpendicular to X and Z (so up stays up). For more info see Representing Points on a Circular Radar Math approach

3. Transfrom B to B' by that matrix

   Again in the QA linked in #1 is how to do it. It is simple matrix/vector multiplication.