I'm learning some shader programming in HLSL, using MikuMikuDance as an engine. I don't have access to the main renderer's code, and I'm limited to shader model 3 under DX9. I'm trying to create a shader that would render an environment to a sphere map, which would be useful for some people, and as a tool to experiment and learn. My naive approach seems to lead to interpolation issues. Additionally, it appears that some vertices are getting clipped inappropriately, leading to polygons not being drawn at the borders.
I thought I would try just running sphere map code in reverse in the vertex shader to create my render:
Pos = mul( Pos, WorldMatrix );
Out.Eye = CameraPosition - Pos.xyz;
Pos = mul(Pos, ViewMatrix);
float far = 300000;
float3 En = normalize(Pos.xyz);
Out.Pos = float4(En.x, En.y, length(Out.Eye)/(far), 1);
This leads to perspective issues, particularly with larger polygons:
That makes sense to me; I'm doing linear interpolation when it should be spherical. However, using nointerpolation, centroid, and/or noperspective on Pos has no apparent effect.
Is there a way to get proper perspective with this unusual projection? Am I going about this the entirely wrong way?
There is no practical way to apply a true spherical transform without modifying the rendering pipeline. This is not an interpolation problem but rather a rasterization and vertex transformation problem. The key, as you have noticed, is that spherical projection inherently requires that straight lines become curved. If the pipeline already utilized tessellation of some sort it might be possible, but otherwise, transforming vertices alone can never make the straight edges between them become curved.
If you are able to also modify or add a screen-space effect (e.g. post-processing), you may be able to manipulate the source texture coordinates to apply the spherical transformation there instead. In this case, it would still be beneficial to modify the vertex shader to increase the FOV so you can fill out more of the hemisphere. This will be a trade-off between FOV and resolution though - as you approach 180 degrees, the resolution difference between pre- and post-projection increases. At 179 degrees, virtually all pixels in the initial render will map to the outer ring of the hemisphere.