Search code examples
three.jsglsltexture-mappingperspectivecamera

Perspective projection of GLSL textureCube on arbitrary geometry from same origin does not conserve straight lines

Problem: Certain straight lines (the blue z axis in this example) are not projected to straight lines when projecting the picture of a rotated cubecamera with the cube shader on a non-cube screen and viewing the screen from the ideal position using a perspective camera. Why? What am I getting wrong?

My understanding of the cube camera is that it consists of 6 perspective cameras with fov=90 and aspect=1, each aligned to a positive or negative main axis, to cover the whole space (between near and far). It can be positioned and rotated freely (and even scaled and skewed, I suppose), and then the generated images will account for this. The images are a CubeTexture, which is more or less just 6 square textures. The "cube size" in the parameters is the resolution of the square textures.

My understanding of the cube shader and of textureCube is that the vector supplied is treated as a direction only, and that the color is obtained by intersecting a ray in that direction (from the center) with the surface of the cube.

When positioning a perspective camera, my understanding is that the position of the camera is the eye position, not the projection plane position or something else.

With this in mind, I would expect that: If the projection surface geometry covers the whole view of the observer (perspective) camera and the observer camera is positioned in the same origin as the cube shader uses, then even if the cube camera is rotated, the picture from the observer camera will be the same for every possible projection surface geometry (except for some minor sampling/aliasing artifacts). Only when the observer camera or cube shader origin were moved, I would expect differences caused by the geometry.

Instead I get this with the image of a rotated CubeCamera projected to a cylinder: The blue z axis is broken (presumably on the edge of the cylinder) Notice how the blue z axis is "broken" (presumably on the edge of the cylinder).

this with the same image projected to a cube: enter image description here

this with the image of an unrotated CubeCamera projected to a cylinder: enter image description here

and this with the same image projected to a cube: enter image description here

The cropping is different for each image (done manually), but the observer camera settings and rotation are the same. It looks like the unrotated CubeCamera gives the same image for the cube and the cylinder, while the rotated CubeCamera gives very different images.

Sufficient code to demonstrate the problem is enclosed below. I have also uploaded a running version here.

<html>
<head>
    <title>Projectors problem test</title>
    <script src="three.js-master/build/three.js"></script>
</head>

<body>
<script>
"use strict";

//Globals
var renderer, simulatedScene, simulationCubeCamera, idealSimulatorScene, camera;

(function main() {
    //Renderer setup
    document.body.style = "overflow: hidden;";
    var container = document.createElement("div");
    container.style = "position: absolute; top: 0; left: 0;"
    document.body.appendChild(container);
    renderer = new THREE.WebGLRenderer({antialias: true});
    renderer.setSize(window.innerWidth, window.innerHeight);
    container.appendChild(renderer.domElement);

    //Setup of simulated scene: 
    simulatedScene = new THREE.Scene();

    simulatedScene.add(new THREE.AxisHelper(500));

    //Cubecamera setup
    simulationCubeCamera = new THREE.CubeCamera(5, 1000, 2048);
    simulationCubeCamera.position.set(0,50,0);
    //Comment out CubeCamera rotation to see the difference:
    simulationCubeCamera.rotation.z = -0.25*Math.PI;
    simulatedScene.add(simulationCubeCamera);
    simulationCubeCamera.update(renderer, simulatedScene);  

    //Define (arbitrary mesh-based) projector screen geometry:
    //Box geometry seems to work. Cylinder geometry works(?) for non-rotated     CubeCamera:
    var screenGeometry = /*new THREE.BoxBufferGeometry(5,5,5, 1,1,1);*/new     THREE.CylinderBufferGeometry(5,5,5, 12288, 1);

    //Make "ideal" projection on the screen geometry using cube shader:
    idealSimulatorScene = new THREE.Scene();
    let cubeShader = THREE.ShaderLib.cube;
    cubeShader.uniforms.tCube.value =     simulationCubeCamera.renderTarget.texture;
    let idealScreenMat = new THREE.ShaderMaterial({
        uniforms: cubeShader.uniforms,
        vertexShader: cubeShader.vertexShader,
        fragmentShader: cubeShader.fragmentShader,
        //depthWrite: false,
        side: THREE.BackSide});

    var idealProjectorScreen = new THREE.Mesh(
        screenGeometry,
        idealScreenMat  
    );
    idealSimulatorScene.add(idealProjectorScreen);

    //Observer camera setup:
    camera = new THREE.PerspectiveCamera(90, window.innerWidth /     window.innerHeight, 0.01, 15);
    camera.lookAt(new THREE.Vector3(5,-50,35));

    renderer.render(idealSimulatorScene, camera);
})();

</script>
</body>
</html>
Solution

  • I solved it. It turns out that the cube shader is specialized for displaying a sky cube. Ironically, the solution was a simplification. I just had to use the world position instead of transformDirection:

    Now my vertex shader looks like this (after substituting some macros, function calls and intermediate variables):

    varying vec3 vWorldPosition;

void main() {
    //With transformDirection:
    //vWorldPosition = normalize( ( modelMatrix * vec4( position, 0.0 ) ).xyz );
    //With just the world position (interpreted by the textureCube as a direction)
    vWorldPosition = ( modelMatrix * vec4( position, 1.0 ) ).xyz;

    gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
}

    Notice that the normalization is removed too. textureCube handles unnormalized vectors, and in fact the interpolated vectors would not in general be normalized anyway.

    I can reuse the old fragment shader.