WebGL: using framebuffers for picking multiple objects

I am trying to implement picking in WebGL. I have a lot of objects (around 500), and I'd like each one to be allowed to be picked. In order to do that, I did a loop which assigns a unique colour to each object (cf. picking principle):

for (var i = 0, len = objects.length; i < len; i++) {
   framecolors[count++] = i % 256 / 256; //Red
   framecolors[count++] = Math.floor(i/256) / 256; //Green
   framecolors[count++] = Math.floor(i/(256*256)) / 256; //Blue
}

framecolors was then used in a classical buffer to check whether each object had a different shade of red. It worked.

Now, I want to use my objects' original colours, AND a framebuffer with the shades of red in the background. I've been through some code, and I am a bit confused.

Here's what I have tried so far.

Function called before picking:

//Creates texture
colorTexture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, colorTexture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 400, 400, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);

//Creates framebuffer
fb = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, fb);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, colorTexture, 0);
gl.bindTexture(gl.TEXTURE_2D, colorTexture);
gl.enable(gl.DEPTH_TEST);

gl.bindFramebuffer(gl.FRAMEBUFFER, fb);
gl.clear(gl.DEPTH_BUFFER_BIT);
gl.drawArrays(gl.POINTS, 0, vertexPositionBuffer.numItems);

Function called after:

gl.bindFramebuffer(gl.FRAMEBUFFER, null);
gl.bindTexture(gl.TEXTURE_2D, colorTexture);
gl.drawArrays(gl.POINTS, 0, vertexPositionBuffer.numItems);

As you can probably understand, I am not very comfortable with framebuffers, I just don't really get how they work, even though I read a lot about them. I have no idea how I can link framecolors to the framebuffer. Is there a way?

Thanks, R.

Solution

A framebuffer is a collection of attachments (renderbuffers and/or textures). It works just like rendering without a framebuffer. (in fact the browser is internally using a framebuffer to implement WebGL's canvas)

In your case you're missing a few things. You most likely need a depth buffer attached otherwise when you render your scene you won't get zBuffering and the wrong objects will appear in front.

// create renderbuffer
depthBuffer = gl.createRenderbuffer();
gl.bindRenderbuffer(gl.RENDERBUFFER, depthBuffer);

// allocate renderbuffer
gl.renderbufferStorage(
      gl.RENDERBUFFER, gl.DEPTH_COMPONENT16, width, height);  

// attach renderebuffer
gl.framebufferRenderbuffer(
      gl.FRAMEBUFFER,
      gl.DEPTH_ATTACHMENT,
      gl.RENDERBUFFER,
      depthBuffer);

In general you should also check your framebuffer works. After attaching all the attachments you call gl.checkFramebufferStatus

if (gl.checkFramebufferStatus(gl.FRAMEBUFFER) != gl.FRAMEBUFFER_COMPLETE) {
   alert("this combination of attachments does not work");
   return;
}

A framebuffer can be incomplete for any number of reasons. The most common is the attachments are not same size or the GPU doesn't support that combination of attachments. Note: In WebGL certain combinations are required to work but you since you might change the code later to use different formats it's probably still a good idea to check.

You also need to set the viewport by calling gl.viewport whenever you switch framebuffers.

 gl.bindFramebuffer(gl.FRAMEBUFFER, someFramebuffer);
 gl.viewport(0, 0, someFramebufferWidth, someFramebufferHeight);

That includes putting it back when setting things back to the canvas

 gl.bindFramebuffer(gl.FRAMEBUFFER, null);  // render to canvas
 gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);

Finally there's a bug in the code above in that you're only clearing the depth buffer of the framebuffer where you call gl.clear. You want to call

 gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

Since you're going to read the colors later otherwise old colors will be left over.

Finally, and I guess you know this. You figure out which pixel corresponds to the mouse click and call

 var colorPicked = new Uint8Array(4);
 gl.readPixels(pickX, pickY, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, colorPicked);

Note that when you call gl.readPixels you must have your framebuffer bound with gl.bindFramebuffer or gl.readPixels will read from the canvas.