Unity3D, have BOTH 3D and UI Raycaster simultaneously respond to touch?

Question

These days in Unity you very easily detect touch on objects using a raycaster...

public class DragThreeDee:MonoBehaviour,IPointerDownHandler
       IPointerUpHandler, IBeginDragHandler,
       IDragHandler, IEndDragHandler
       {
       public void OnPointerDown (PointerEventData data)
          { etc

So, regarding your UI layer, it correctly ignores touch on the UI layer and so on.

What if you want the following to happen,

you want to collect (simultaneously, in the same frame, both) a touch which is over both a 3D object, and, a UI item?

(Imagine you have, say, a few transparent UI buttons on the screen; you also have ordinary 3D robots running around. User clicks on a point on the screen where there's a UI button, and "underneath", a robot. BOTH the robot's script as above, and the button, should respond.)

How would you do that? I tried extensively using separate cameras, separate EventSystem, adjusting the layers, blocking masks and so on. I can't find a way to do it.

How to do this in Unity?

Answer 1

Looks complicated but can be done.

1.Route the data from PointerEventData in all of the UI components you want to unblock 3D GameObjects raycast on.

2.Get all the instance of PhysicsRaycaster with FindObjectsOfType<PhysicsRaycaster>(). In terms of performance, it would make sense to cache this.

3.Perform a Raycast with PhysicsRaycaster.Raycast which will return all GameObjects with Collider attached to it.

4.Use ExecuteEvents.Execute to send the appropriate event to the result stored in the RaycastResult.

RaycastForwarder script:

public class RaycastForwarder : MonoBehaviour
{
    List<PhysicsRaycaster> rayCast3D = new List<PhysicsRaycaster>();
    List<RaycastResult> rayCast3DResult = new List<RaycastResult>();

    private static RaycastForwarder localInstance;
    public static RaycastForwarder Instance { get { return localInstance; } }


    private void Awake()
    {
        if (localInstance != null && localInstance != this)
        {
            Destroy(this.gameObject);
        }
        else
        {
            localInstance = this;
        }
    }

    public void notifyPointerDown(PointerEventData eventData)
    {
        findColliders(eventData, PointerEventType.Down);
    }

    public void notifyPointerUp(PointerEventData eventData)
    {
        findColliders(eventData, PointerEventType.Up);
    }

    public void notifyPointerDrag(PointerEventData eventData)
    {
        findColliders(eventData, PointerEventType.Drag);
    }

    private void findColliders(PointerEventData eventData, PointerEventType evType)
    {
        UpdateRaycaster();

        //Loop Through All Normal Collider(3D/Mesh Renderer) and throw Raycast to each one
        for (int i = 0; i < rayCast3D.Count; i++)
        {
            //Send Raycast to all GameObject with 3D Collider
            rayCast3D[i].Raycast(eventData, rayCast3DResult);
            sendRayCast(eventData, evType);
        }
        //Reset Result
        rayCast3DResult.Clear();
    }

    private void sendRayCast(PointerEventData eventData, PointerEventType evType)
    {
        //Loop over the RaycastResult and simulate the pointer event
        for (int i = 0; i < rayCast3DResult.Count; i++)
        {
            GameObject target = rayCast3DResult[i].gameObject;
            PointerEventData evData = createEventData(rayCast3DResult[i]);

            if (evType == PointerEventType.Drag)
            {
                ExecuteEvents.Execute<IDragHandler>(target,
                                        evData,
                                        ExecuteEvents.dragHandler);
            }

            if (evType == PointerEventType.Down)
            {
                ExecuteEvents.Execute<IPointerDownHandler>(target,
                               evData,
                               ExecuteEvents.pointerDownHandler);
            }

            if (evType == PointerEventType.Up)
            {
                ExecuteEvents.Execute<IPointerUpHandler>(target,
                    evData,
                    ExecuteEvents.pointerUpHandler);
            }
        }
    }

    private PointerEventData createEventData(RaycastResult rayResult)
    {
        PointerEventData evData = new PointerEventData(EventSystem.current);
        evData.pointerCurrentRaycast = rayResult;
        return evData;
    }

    //Get all PhysicsRaycaster in the scene
    private void UpdateRaycaster()
    {
        convertToList(FindObjectsOfType<PhysicsRaycaster>(), rayCast3D);
    }

    private void convertToList(PhysicsRaycaster[] fromComponent, List<PhysicsRaycaster> toComponent)
    {
        //Clear and copy new Data
        toComponent.Clear();
        for (int i = 0; i < fromComponent.Length; i++)
        {
            toComponent.Add(fromComponent[i]);
        }
    }

    public enum PointerEventType
    {
        Drag, Down, Up
    }
}

RayCastRouter script:

public class RayCastRouter : MonoBehaviour, IPointerDownHandler,
       IPointerUpHandler,
       IDragHandler
{
    public void OnDrag(PointerEventData eventData)
    {
        RaycastForwarder.Instance.notifyPointerDrag(eventData);
    }

    public void OnPointerDown(PointerEventData eventData)
    {
        RaycastForwarder.Instance.notifyPointerDown(eventData);
    }

    public void OnPointerUp(PointerEventData eventData)
    {
        RaycastForwarder.Instance.notifyPointerUp(eventData);
    }
}

Usage:

A.Attach RaycastForwarder to an empty GameObject.

B.Attach RayCastRouter to any UI component such as Image that you don't want to block a 3D GameObject. That's it. Any UI component with RayCastRouter attached to it will be able to allow a 3D GameObject behind it receive a raycast.

An event will now be sent to 3D Object that has a script that implements functions from IPointerDownHandler, IPointerUpHandler and IDragHandler interface.

Don't forget to attach Physics Raycaster to the camera.