Issue with transforming a 3d point to screen coordinates

Question

I have a point cloud in 3d world space that I would like to render to the screen by projecting it's individual points using an MVP matrix. But something about the projection seems off. There appear to be 2 copies of the point cloud mirrored accross all axis, so moving in opposing directions:

To debug the issue, I've tried only rendering one pixel and fixing that point to (0, 0, 0). The pixel still appears twice in opposing directions:

Here's the pixel with the camera positioned above it (0, 1, 0), showing up above and below it:

Here's the C# script I'm using to calculate the MVP matrix, dispatch the shader and blit to the output texture:

using UnityEngine;

public class MultiPass : MonoBehaviour
{
    public Texture2D input;
    public ComputeShader computeShader;
    public Vector2Int projectionResolution;

    public RenderTexture transformed;
    private Camera mainCamera;
    private int project, projectGroupsX, projectGroupsY;

    private void Start()
    {
        mainCamera = GetComponent<Camera>();

        project = computeShader.FindKernel("Project");

        computeShader.GetKernelThreadGroupSizes(project, out uint threadGroupSizeX, out uint threadGroupSizeY, out _);
        projectGroupsX = projectionResolution.x / (int)threadGroupSizeX;
        projectGroupsY = projectionResolution.y / (int)threadGroupSizeY;

        transformed = new RenderTexture(projectionResolution.x, projectionResolution.y, 0);
        transformed.enableRandomWrite = true;
        transformed.format = RenderTextureFormat.RGFloat;

        computeShader.SetFloat("PI", Mathf.PI);
        computeShader.SetFloat("PI2", Mathf.PI * 2);
        computeShader.SetVector("INPUT_RESOLUTION", new Vector2(input.width, input.height));
        computeShader.SetVector("PROJECTION_RESOLUTION", new Vector2(projectionResolution.x, projectionResolution.y));
        computeShader.SetTexture(project, "Input", input);
        computeShader.SetTexture(project, "Transformed", transformed);
    }

    private void Update()
    {
        RenderTexture rt = RenderTexture.active;
        RenderTexture.active = transformed;
        GL.Clear(false, true, Color.clear);
        RenderTexture.active = rt;

        Matrix4x4 MVP = GL.GetGPUProjectionMatrix(mainCamera.projectionMatrix, true) * mainCamera.worldToCameraMatrix;
        computeShader.SetMatrix("MVP", MVP);
        computeShader.Dispatch(project, projectGroupsX, projectGroupsY, 1);
    }

    private void OnRenderImage(RenderTexture source, RenderTexture destination)
    {
        Graphics.Blit(transformed, destination);
    }
}

And here's the shader that multiplies the points with the matrix:

#define NCLIP 1
#define FCLIP 30
#define MAP_TO_RANGE(tc, targetRange) \
    ((tc) * ((targetRange) - 1))
#define NORMALIZE_RANGE(tc, sourceRange) \
    ((tc) / ((sourceRange) - 1))

Texture2D<float4> Input;
RWTexture2D<float2> Transformed;

float2 INPUT_RESOLUTION, PROJECTION_RESOLUTION;
float PI, PI2;
float4x4 MVP;


#pragma kernel Project

[numthreads(8, 8, 1)]
void Project(uint3 id : SV_DispatchThreadID)
{
    // 1. calculate some points (the depth based point cloud or just a sphere)
    float2 uv = NORMALIZE_RANGE(id.xy, PROJECTION_RESOLUTION);
    float d = Input[MAP_TO_RANGE(uv, INPUT_RESOLUTION)].a;
    
    float2 ll1 = uv.yx;
    ll1.x *= PI;
    ll1.y *= PI2;
    ll1.y += PI;

    float CP = d * (FCLIP - NCLIP) + NCLIP;
    
    // override point cloud with sphere
    // CP = FCLIP;

    float3 P = float3(
        CP * sin(ll1.y) * sin(ll1.x),
        CP * cos(ll1.x),
        CP * cos(ll1.y) * sin(ll1.x));
    
    // for only transforming one point
    /*
    if (!any(id.xy))
        P = float3(0, 0, 0);
    else
        return;
    */
    

    // 2. the actual matrix multiplication
    float4 pos4 = float4(P, 1);
    float4 transformedPosition = mul(MVP, pos4);
    
    float2 screenCoords;
    screenCoords = (transformedPosition.xy / transformedPosition.w) * 0.5 + 0.5;
    screenCoords = MAP_TO_RANGE(screenCoords, PROJECTION_RESOLUTION);
    
    Transformed[screenCoords] = float2(1, 1);
}

I am absolutely at a dead end here. I can't image the point generation being a problem, as the issue persists even if I hardcode in just one point at (0, 0, 0). I also can't imagine the issue being with the conversion to screen/texture coordinates, as an issue there would propably show in the form of a distorted or even upside-down image, but not 2 sets of points accurately moving in 3d space. So the only thing left is the projection matrix, I guess. But I tried pretty much all ways of constructing the matrix I've been able to find online, and none of them affected the issue. So I would be extremely grateful for any ideas on what the issue with the matrix multiplication I'm doing could be or on other steps I could take to try and debug this. Also, if there are any other aspects of the scripts that could be causing the issue, I'd be happy to know so I can look into it.

Btw sorry for the poor quality, I don't think I can embed anything apart from gif's here. If anyone would like more detailed files, please let me know

Answer 1

I've been able to figure it out by further debugging a single matrix multiplication on the cpu. Turns out that, if something is behind the camera, it gets rendered as well, but in this mirrored way. So basically, resolving the issue was about detecting whether something's in front or behind the camera. I was able to differenciate by checking the w component of the transformed vector. Adding a check like this resolved the issue:

if (transformedPosition.w <= 0)
{
    return;
}

Also don't forget a check like this when converting to screen space:

if (screenCoords.x < 0 || screenCoords.y < 0)
{
    return;
}