MRTK V2 - Spectator View: Can't see Dynamic Textures

Question

Setup:
I'm using the nativ render plugin to display webcontant on a texture and not overload the main thread. Unity side passes a texture into the plugin and the code changes the pixels of it each frame. This is done by using Texture.GetNativeTexturePtr:

private void CreateTextureAndPassToPlugin()
{
    // Create a texture
    Texture2D tex = new Texture2D(1280, 720);
    tex.filterMode = FilterMode.Point;

    // Call Apply() so it's actually uploaded to the GPU
    tex.Apply();

    // Set texture onto material
    GetComponent<Renderer>().material.mainTexture = tex;

    // Pass texture pointer to the plugin
    SetTextureFromUnity (connID, tex.GetNativeTexturePtr(), tex.width, tex.height);
}

I set up the whole specator view stuff according to the guide from Microsoft. I can do stuff with my HoloLens and a second person can see my interaction via an android device. The problem is that the spectator can't see on the android device my texture/webcontent.

What I want:
Does sb know how to tell unity that the texture has been changed and needs to be updated? My knowledge is very limited in this area. As far as I can understand, the problem seems to be the spectator and the spectator does not provide a way to fix the problem.

My Specs:
Unity 2019.2.0f1
Visual Studio 2017
Native Plugin: Unity Bitbucket Github

Answer 1

Spectator View does not currently have built in support for dynamic textures.

If you are trying to film an experience compared to provide a second view point, a less than ideal workaround would be to fake this content in a video. You could theoretically record this texture content to an mp4 on the hololens device, then replace your content with an mp4 for filming purposes.

If you need the content to update in real time, you might be able to use something like the MixedReality-WebRTC repo to take the content and synchronize it to the android device. I have not personally spent a great deal of time using WebRTC and am not sure the associated cost with doing this.

Regardless, this is a reasonable feature request, I've opened the following issue for tracking: https://github.com/microsoft/MixedReality-SpectatorView/issues/224