Search code examples
reactjscanvasthree.jsreact-three-fiberreact-three-drei

react-three-fiber: How to capture events ANYWHERE in <Canvas />?

I'm trying to create a 3D experience using react-three-fiber where the user can interact with any part of the 3D canvas, including points on the canvas that both (a) intersect an object, and (b) where there is nothing intersecting the ray at all.

This means I can't use the pointer events abstractions built into <mesh /> etc., as those pointer events ONLY capture points that intersect objects.

I tried attaching handlers to the <Canvas onMouseDown={...} />, however I'm unable to access the THREE internals from there as the event passed by onMouseDown does not contain them, and the useThree hook must be deeper in the tree to access the THREE react context.

So I also tried creating a component nested inside <Canvas /> (see below) where I'd be able to use the useThree hook:

<Canvas>
  <MouseHandler>
    ...
    <mesh />
    ...
  </MouseHandler>
</Canvas>

export function MouseHandler({ children }) {
   const { Camera } = useThree()
   return <Html><div onMouseDown={...}>{children}</div></Html>
}

...but then react-three-fiber complains that I have THREE objects inside HTML objects.

Anyone have any suggestions on how else I might be able to solve this?

Solution

  • Doesn't feel like a common enough use case that react-three-fiber would ever support, so what I ended up doing was hack it like this, using React Refs to yank a "pointer" to the three objects I needed from deeper into the tree.

    const threeRef = useRef()

<MouseHandler threeRef={threeRef}>
  <Canvas>
    <ThreeConnect threeRef={threeRef}>
      ...
      <mesh />
      ...
    </ThreeConnect>
  </Canvas>
</MouseHandler>

...

export function MouseHandler({ threeRef, children }) {
   const { Camera } = threeRef.current
   return <Html><div onMouseDown={...}>{children}</div></Html>
}

...

export default function ThreeConnect({ children, threeRef }) {
  const { scene, camera } = useThree()

  useEffect(() => {
    threeRef.current = {
      ...threeRef.current,
      camera,
      scene,
    }
  }, [scene, camera])

  return <>{children}</>
}