How can I get the coordinates of a point in 3D based on a mouse click?

Question

I want that when I click on a specific point on the screen, a function call that return the point in 3d scene that I click.

For example, when I click on the top left of the screen, it should return x = 0, y = 1, z= 1; please help me create a method to do this.

edit:

Root = new BranchGroup();

setLayout(new BorderLayout());
add(canvas3D,BorderLayout.CENTER);
SimpleUniverse universe = new SimpleUniverse(canvas3D);

Shape();
universe.addBranchGraph(Root);
ViewingPlatform viewingPlatform = universe.getViewingPlatform();
OrbitBehavior behavior = new OrbitBehavior(canvas3D);
behavior.setSchedulingBounds(bounds);
viewingPlatform.setViewPlatformBehavior(behavior);
viewingPlatform.setNominalViewingTransform();
}

I'm using SimpleUniverse in NetBeans.

Answer 1

I'm afraid the answer isn't straight forward. Depending on what is in your scene the mouse coordinates should change when you click on the screen. For instance if you have two objects so that the front one occludes the back one then you'd want to get the front objects coordinates. This is called mouse picking and there are several techniques for it. I found a few forums that explain how it's done and have code examples.

Basically the idea is that you imagine there being a laser (or something else that casts a ray) between you the user and the screen. This thing then projects a ray through the point on the screen surface where the mouse was clicked "into" the screen. Anything that is on the rays path will then be picked and optionally the occlusion order of objects in the ray's path is resolved to give you the object closest to the "screen".

I am not familiar with J3D, but scraped the following code together from a few tutorials. It should get you started at least. The this you are looking for is this line Point3D intercept = ... at the bottom.

http://www.java3d.org/selection.html

package j3d_picking;

import java.awt.*;
import javax.swing.*;
import javax.media.j3d.*;
import com.sun.j3d.utils.universe.*;
import com.sun.j3d.utils.picking.behaviors.*;
import com.sun.j3d.utils.geometry.*;
import com.sun.j3d.utils.picking.PickIntersection;
import com.sun.j3d.utils.picking.PickResult;
import com.sun.j3d.utils.picking.PickTool;
import javax.vecmath.Point3d;

public class HelloJava3D
        extends JFrame
{

    public HelloJava3D()
    {
        GraphicsConfiguration config = SimpleUniverse.getPreferredConfiguration();
        Canvas3D canvas3D = new Canvas3D(config);

        BranchGroup scene = createSceneGraph();

        // SimpleUniverse is a Convenience Utility class
        SimpleUniverse simpleU = new SimpleUniverse(canvas3D);

        // This moves the ViewPlatform back a bit so the
        // objects in the scene can be viewed.
        simpleU.getViewingPlatform().setNominalViewingTransform();

        BoundingSphere behaveBounds = new BoundingSphere();
        ExamplePickBehavior behavior = new ExamplePickBehavior(canvas3D, scene, behaveBounds);
        scene.addChild(behavior);

        scene.compile();
        simpleU.addBranchGraph(scene);

        getContentPane().add(canvas3D, BorderLayout.CENTER);
    } // end of HelloJava3D (constructor)

    public BranchGroup createSceneGraph()
    {
        // Create the root of the branch graph
        BranchGroup objRoot = new BranchGroup();

        // Create a simple shape leaf node, add it to the scene graph.
        // ColorCube is a Convenience Utility class
        ColorCube cube = new ColorCube(0.4);
        cube.setCapability(Node.ENABLE_PICK_REPORTING);
        PickTool.setCapabilities(cube, PickTool.INTERSECT_FULL);
        objRoot.addChild(cube);

        return objRoot;
    } // end of createSceneGraph method of HelloJava3D

    public static void main(String[] args)
    {
        JFrame frame = new HelloJava3D();
        frame.setTitle("Hello Java3D");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setBounds(0, 0, 400, 300);
        frame.setVisible(true);
    }

    private class ExamplePickBehavior extends PickMouseBehavior
    {

        public ExamplePickBehavior(Canvas3D canvas, BranchGroup bg, Bounds bounds)
        {
            super(canvas, bg, bounds);
            setSchedulingBounds(bounds);

            pickCanvas.setMode(PickTool.GEOMETRY_INTERSECT_INFO);
            // allows PickIntersection objects to be returned
        }

        public void updateScene(int xpos, int ypos)
        {
            pickCanvas.setShapeLocation(xpos, ypos);
            // register mouse pointer location on the screen (canvas)

            Point3d eyePos = pickCanvas.getStartPosition();
            // get the viewer's eye location

            PickResult pickResult = null;
            pickResult = pickCanvas.pickClosest();
            // get the intersected shape closest to the viewer

            if (pickResult != null) {
                PickIntersection pi = pickResult.getClosestIntersection(eyePos);
                // get the closest intersect to the eyePos point
                Point3d intercept = pi.getPointCoordinatesVW();
                System.out.println(intercept);
                // extract the intersection pt in scene coords space
                // use the intersection pt in some way...
            }
        } // end of updateScene(  )
    } // end of ExamplePickBehavior class
}