If you have a given vertex how could you determine whether that vertex is to the left or right (or possibly directly in line with) the camera? All the info I find on this shows how to do it in 2D but I need 3D.
If I add the camera's look vector onto the camera's location I will have the ray that I need. But the up vector also has to be thrown into the equation, so it seems a little tricky.
I think I have to find a transformation that makes the up vector = (0, 1, 0) and the look vector = (0, 0, 1) and then apply that transformation to the vertex. Then you can just say if the vertice's x coordinate is less than the camera's then it is to its left, else it is to the right.
It's as simple as multiplying the vector by the View-Projection matrix. Here is what it boils down to though so you don't have to do a full matrix multiplication (you only need to check one section of the resulting matrix to see which side of the screen the vertex is on)
private bool left(Vector3 v)
{
if (viewProjection.M11 * v.X + viewProjection.M21 * v.Y + viewProjection.M31 * v.Z + viewProjection.M41 < 0)
return true;
return false;
}
viewProjection is obviously just the view matrix * projection matrix