MRTK Add ManipulationHandler in C#

Question

I'm attempting to dynamically add Manipulation Events to a ManipulationHandler that is being added to child objects of a parent. The parent object will be what the user is inspecting, but the user will be able to grab parts off of the parent and inspect them more closely. (i.e. you can look at an engine (parent object), but if you want to inspect the pistons (child objects) you can grab them and look at them)

Instead of having to go into every child object and manually add it in Unity I'd like to be able to add the parent object and just procedurally add the ManipulationHandler and ManipulationEvents on start or awake.

So far I have the following code for adding the ManipulationHandler script, but to add the ManipulationEvent I'm not sure how to set up the pointers so I can use the script and function I want from the source:

gameObject.AddComponent<ManipulationHandler>();
ManipulationHandler handler = gameObject.GetComponent<ManipulationHandler>();
ManipulationEvent newevent = new ManipulationEvent();
ManipulationEventData eventdata = new ManipulationEventData();
eventdata.ManipulationSource = gameObject;

The program works when I grab the objects, but I'd like to add manipulation events when I grab them so I can display additional data.

I see there's a getter and setter for Pointer in ManipulationEventData, but I'm not sure how to instantiate IMixedRealityPointer and how to get it to work. I'm also not sure if that's the object I actually need to accomplish what I'd like to accomplish.

I apologize in advance if I've missed something obvious. I'm new to MRTK.

Thanks!

Answer 1

The ManipulationHandler has four callback events among them OnManipulationStarted and OnManipulationEnded you can simply add listeners to. (see UnityEvent.AddListener)

Unless I understood your question wrong you don't have to instantiate any IMixedRealityPointer. You don't create the event data yourself but rather the ManipulationHandler feeds these events with the current event data including the interacting pointer information. The ManipulationHandler uses OnPointerDown and OnPointerDragged and OnPointerUp via the IMixedRealityPointerHandler interface in order to manage the interacting pointers and invokes the according events where needed.

Instead of using AddComponent followed by GetComponent directly store and use the return value of AddComponent which will be the reference of the newly added Component. MRTK also has an extension method

T EnsureComponent<T>(this Component component) where T : Component

so that you can simply use e.g.

var handler = this.EnsureComponent<ManipulationHandler>();

which internally first checks whether the component already exists, and if not it adds it.

Note that in order to enable near interactions you will also need a NearInteractionGrabbable so you should add this one too.

You also will have to make sure that your objects have some sort of Collider attached to the same GameObject as the NearInteractionGrabbable.

    ...
    gameObject.transform.EnsureComponnet<NearInteractionGrabbable>();
    var handler = gameObject.transform.EnsureComponnet<ManipulationHandler>();

    handler.OnManipulationStarted.AddListener(HandleOnManipulationStarted);
    handler.OnManipulationEnded.AddListener(HandleOnManipulationEnded);
    ...

/// <summary>
/// If you need it later you need to store the pointer since unfortunately in 
/// OnManipulationEnded the <see cref="ManipulationEventData.Pointer"/> is null
/// (no idea why they do it this way :D )
/// </summary>
private IMixedRealityPointer _pointer;    

private void HandleOnManipulationStarted(ManipulationEventData eventData)
{
    _pointer = eventData.Pointer;

    // whatever shall happen when manipulation started
}

private void HandleOnManipulationEnded(ManipulationEventData eventData)
{
    // whatever shall happen when manipulation ended
}

---

Note: I am not sure if this thing you are trying to achieve is possible with this architecture ... it is very possible that nesting various ManipulationHanlder leads to strange behavior here and there. Especially very small parts will be almost impossible to grab ...