I am working on a webgame which requires selection of drawn objects, so not in simple geometric shapes. To do this, I have been drawing them as sprites in a scene on a canvas using Three.JS. I have been racking my brain for months trying to figure out how to determine if the ray of the raycaster in my webgl context (using ThreeJS) is colliding with the transparent part of a sprite. I've searched and read as many posts as I could find, but only two solution posts I've found are:
var ctx =renderer.getContext("experimental-webgl", {preserveDrawingBuffer: true})
const pixel = new Uint8Array(4)
ctx.readPixels(10,30,1,1, ctx.RGBA, ctx.UNSIGNED_BYTE, pixel); log(pixel);
which requires using preserveDrawingBuffer, which would destroy my performance, or:
raycaster.intersectObject(scene.getObjectByName('some sprite')[0]].uv
and then using math wizardry to compare the UV coordinates with the image itself, I guess loaded into a buffer somewhere from scene.getObjectByName('some sprite').material.map.image.src.valueOf(), and then if the matching pixel or fuzzy area is transparent then we don't consider the sprite selected. This would be additionally difficult because the sprites are often rotated, scaled, center-offset, etc. I am not even 100% sure how I would implement that at all.
I am wondering if there is any other way, particularly a more 'proper' way to do this. Is there some way I can convert sprite images into/supply my images with some kind of meshes, then paint the mesh with the image precisely, so that the raytracer will actually have to use the correct object shape instead of just an image plane (sprite)? Would it be better to take every sprite and painstakingly model its edges in a 3d modeling program? I have tried every combination of "mesh," "geometry," "image," "sprite," etc and found no solution.
Any help is appreciated. Beggars can't be choosers, but I am also hoping to have animated objects eventually, so it would be additionally helpful if that was not rendered impossible by this solution.
Thank you
e: see @gman's comment below. I worked in the color picking system, only really changing emissive.setHex to material.color.set(id), then checked for color instead of emissives. works great, but seems to be slowing things down a bit. Will have to see if I implemented it badly or if I can make it more efficient. Thanks to you both!!
With an additional draw pass, you can render the scene into a texture with a dedicated material for every object, and pick the 2D point into it
You may create a basic material with a color matching every object's ID as described in this OpenGL hack (convert it to WebGL)
Use Scene.overrideMaterial to assign a material to all the objects and Object3D.onBeforeRender to set the color of each object before drawing them
It will have a cost, but I think it's worth a try