Three.js using WebRTC and applying a Shader

Question

I can't figure out how to apply a shader to a three.js object that has a video texture.

I've been playing around with webRTC and three.js and successfully mapped a video texture onto a mesh using a standard material:

        var material    = new THREE.MeshBasicMaterial({
            color   : 0xffffff,
            map : videoTexture
        });

I want to take it one step further by applying a shader (for this example a sobel shader) to this texture. My attempt is here: http://jsfiddle.net/xkpsE/1/

I'm receiving a bunch of INVALID_OPERATION warnings, but am having trouble understanding how to debug the issue. I also haven't seen anyone else do this so I think it would be beneficial for this knowledge to be public :)

Any help would be appreciated.

Answer 1

You were close. Here it is: http://jsfiddle.net/82fJh/1/. It works under Chrome on OS X, at least.

You had some shader uniforms formatting errors, and you needed to pass the uv's as a varying.

var sobelShader = {
    uniforms: {
        'texture': {
            type: 't',
            value: videoTexture
        },
         'width': {
            type: 'f',
            value: 320.0
        },
         'height': {
            type: 'f',
            value: 240.0
        }
    },
    vertexShader: [
        'varying vec2 vUv;',
        'void main() {',
           'vUv = uv;',
           'gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );',
        '}'
        ].join('\n'),
    fragmentShader: [
        'uniform sampler2D texture;',
        'uniform float width;',
        'uniform float height;',
        'varying vec2 vUv;',
        'void main(void) {',
            'float w = 1.0/width;',
            'float h = 1.0/height;',
            'vec2 texCoord = vUv;',
            'vec4 n[9];',
            'n[0] = texture2D(texture, texCoord + vec2( -w, -h));',
            'n[1] = texture2D(texture, texCoord + vec2(0.0, -h));',
            'n[2] = texture2D(texture, texCoord + vec2(  w, -h));',
            'n[3] = texture2D(texture, texCoord + vec2( -w, 0.0));',
            'n[4] = texture2D(texture, texCoord);',
            'n[5] = texture2D(texture, texCoord + vec2(  w, 0.0));',
            'n[6] = texture2D(texture, texCoord + vec2( -w, h));',
            'n[7] = texture2D(texture, texCoord + vec2(0.0, h));',
            'n[8] = texture2D(texture, texCoord + vec2(  w, h));',
            'vec4 sobel_horizEdge = n[2] + (2.0*n[5]) + n[8] - (n[0] + (2.0*n[3]) + n[6]);',
            'vec4 sobel_vertEdge  = n[0] + (2.0*n[1]) + n[2] - (n[6] + (2.0*n[7]) + n[8]);',
            'vec3 sobel = sqrt((sobel_horizEdge.rgb * sobel_horizEdge.rgb) + (sobel_vertEdge.rgb * sobel_vertEdge.rgb));',
            'gl_FragColor = vec4( sobel, 1.0 );',
        '}'
        ].join('\n')
}

three.js r.53