How to convert screen coordinates to scene coordinates

Question

I created a a-scene with some objects to drag. The final purpose is exactly what aframe-click-drag-component does. Unfortunately, this component is not compatible with the last version of A-Frame.

I created a custom component.

AFRAME.registerComponent('draggable', {
    init: function () {
        /* Some code */
    }
});

I use the aframe-mouse-cursor-component to be able to get the mouseenter and mouseleave events on the draggable object, and detect when the mouse position allows the user to select the object.

I added an EventListener on document.body to know when the dragging starts:

document.body.addEventListener('mousedown', function (e) {
    // start dragging
});

I continuously update a global variable to update the mouse position when a mousemove occurs:

document.addEventListener('DOMContentLoaded', function () {
    document.body.addEventListener('mousemove', function (e) {
        window.mouseX = e.clientX;
        window.mouseY = e.clientY;
    });
});

This way, I can easily get the position of the mouse during the dragging. But I do not know how to convert the position of the mouse on the client to a position in the Virtual Reality (restricted to a 2D plan to make it possible).

I solved this issue by using the raycaster coming from the cursor in the middle of the a-camera, but I want to drag the objects with the mouse-cursor, and this component does not have a raycaster.

I also tried to use some maths to convert the mouse coordinates to a coordinates set relative to the camera, without success (essentially because of the screen size which can vary).

What solutions are available? I would like to update the click-drag or the mouse-cursor, but I have no knowledge of THREE.js.

Answer 1

See https://github.com/mayognaise/aframe-mouse-cursor-component or https://github.com/mrdoob/three.js/blob/dev/examples/js/controls/DragControls.js or https://www.npmjs.com/package/aframe-click-drag-component for examples

The main chunk of code is like:

    canvas.addEventListener( 'mousemove', function () {

      var mouse = new THREE.Vector2();

      var rect = canvas.getBoundingClientRect();

      mouse.x = ( (event.clientX - rect.left) / rect.width ) * 2 - 1;
      mouse.y = - ( (event.clientY - rect.top) / rect.height ) * 2 + 1;

      raycaster.setFromCamera( mouse, camera );
    }, false);