iPhone : OpenGL ES : Detecting if you have tapped a object (cube) on screen

Question

I asked a similar question already which got me to where I am now but I really need some help on this one. Its the last thing in my way to completing something cool (in my eyes lol)

I have a 3d world where I can move around and in this world are simple cubes.

Using the function -(CGPoint)getScreenCoorOfPoint:(IMPoint3D)_point3D I can work out where these cubes are in X,Y within the 3D world. But its not based on where I am but where they are with the 3d Area. Using the same function, I can also work out where I am. Also I can find out where someone has tapped on the screen.

How on earth do I map these together so that I can work out if I have clicked one of them?

Surely I need something to work out which way I am facing. People have suggested rendering them off screen and doing something but it went completely over my head. I think of scrapping the function about and building my own based on the 3d coords (somehow)

Code

for (NSDictionary *item in self.players)
{
    int x;
    int z;

    x =  [[item objectForKey:@"posX"] floatValue];
    z =  [[item objectForKey:@"posZ"] floatValue];

    IMPoint3D playerPos;
    playerPos.x = x;
    playerPos.z = z;
    playerPos.y = 1;

    CGPoint screenPositionOfThisCube  ; 
    screenPositionOfThisCube = [self getScreenCoorOfPoint:playerPos];


    #define TUNE 28
    CGRect fingerSquish = CGRectMake(
                                     screenPositionOfThisCube.x - TUNE,
                                     screenPositionOfThisCube.y - TUNE,
                                     TUNE * 2,
                                     TUNE * 2);

    // now check that the POINT OF THE TOUCH
    // is inside the rect fingerSquish

    if ( CGRectContainsPoint( fingerSquish, pos ) )
    {
        NSLog(@"WOOP");
        // YOU HAVE FOUND THE TOUCHED CUBEY.
        // YOU ARE DONE.  this item is the cube being touched.
        // make the cube change color or something to test it.
    }
}

I also trying out gluUnproject with no success (see my other post)

Answer 1

The notion which you're discussing (selecting object on a 3D screen by 2D coords) is referred to as picking.

See my older posts for some solutions:

How would you solve this opengl necessity (in c) involving knowing in which square in a boardgame did the user click?

OpenGL GL_SELECT or manual collision detection?