Search code examples
opengl-esrenderingwebgl360-degreesxml3d

XML3D: Efficient display & change of large textures

I'm using XML3D and sphere model to display 360 photos. The problem is I need to change large textures frequently.

  • What is the optimal way to use XML3D or webgl for rendering 360 photos?
  • How can I optimize the texture switching? Would it be better to have a set of smaller textures? Is there any set of "good practices" for rendering of 360 images?
  • What is the optimal way to achieve blurring effect between two 360
    "sky" textures?
Solution

  • If you were switching between just two or three textures there would be faster alternatives, but with that many textures your only practical option is to load them one after the other by changing the image tag:

    <texture name="diffuseTexture">
   <img id="photo" src="images/photo1.jpg"/>
</texture>

document.querySelector("#photo").src = "images/photo2.jpg";

    If you profile this with large textures (4096x2048) you'll see a huge amount of time being spent in the call to gl.texImage2D, where the image data is decoded, converted and sent to the GPU. Even on a desktop this can result in stuttering, the situation gets a lot worse on mobile.

    Reducing the size of your textures will speed things up a lot. For example dropping them from 4096x2048 to 2048x1024 brings the time spent in gl.texImage2D down from ~53% to ~13% on my desktop.

    A more low level WebGL library might give you some more options here (such as converting the images to Uint8Arrays by hand before giving them to texImage2D) but XML3D wasn't really designed with this in mind. And as long as you're working with big textures you're going to have slowdowns no matter what you do particularly on mobile.

    If you can put your photos into a video instead that would work much better since the decoding and conversion is done on the GPU in that case. Of course that has its own downsides though.

    Fading one texture into another can be done very simply in a shader using a uniform variable:

    uniform float uFade;
...
vec4 color1 = texture2D(texture1, texcoords);
vec4 color2 = texture2D(texture2, texcoords);
gl_FragColor = mix(color1, color2, uFade);

    Blurring involves several steps. First you need to render the above mix into an offscreen FrameBuffer. That becomes an input texture for a horizontal blur shader, the output of that is given to a vertical blur shader and finally the output of that is used as the texture for the sky object during the scene rendering pass. The first three steps should be rendered onto a full screen quad.

    This can be done in XML3D using the RenderInterface and your own RenderPasses, but IMO it's a lot of trouble to go through for an effect that isn't hugely different from a simple fade.