Problem updating position of grabbable GameObject in VR Scene using PUN2 and Unity3D

Question

I'm trying to add PUN2 support to a simple Oculus starter scene (cubes on a table). I was able to run the official Avatar synchronization on top of my basic Photon2 setup.

This is the current sequence:

• Player 1 ( P1 - the one using Quest) creates the room and joins it

• Player 2 (P2 - the one using Unity Player) joins the room

• Player 1 grabs a cube and moves it around

P2 sees the cube moving normally if P1 just drags it horizontally, but if he tries to lift it from the table the cube seems acting crazy from P2 point of view.

It looks like the cube fights with gravity but I'm not totally sure what is going on. You can see an example in this video where there is an alternation between horizontal drag and lifting from the table: https://gofile.io/?c=A9PN41

If I disable gravity on the cube it seems working fine (although the interpolation a bit slow compared to how P1 is moving the object): https://gofile.io/?c=jM4qEE

The point is that I cannot have cubes flying around the scene :)

Using Unity 2019.1 / PUN2 / Oculus Quest

Answer 1

For the records, I found out what is the problem but still working on a complete solution. When P1 manipulates an object (kept in sync across clients using Photon Transform View), P2 sees the cube trying to synchronize its position/rotation while fighting against gravity (creating that weird jumpy behavior).

After an extensive I research I found out a possible approach which could solve the problem:

• Add a Photon Rigid Body View (sync velocity and angular velocity)
• Add an RPC (activated during manipulation) which:
  • Disables gravity
  • Enables isKinematic
  • Transfer ownership to the Player which is manipulating the object.

This needs to be fully tested on both clients (Master/Normal), but so far the results seem promising.