I'm working with a stereo pair of photos, and trying to obtain a set of 3D points from matching points on these 2 images. How exactly is the 3D point triangulated? Do you throw a ray from the camera pinhole centerpoint? Or do you throw a ray perpendicular to the photos?
You should throw the ray from the camera center point.
Using parallel rays would only be appropriate if the original images were created with an orthographic projection.