Search code examples
unity-game-enginerenderingchromiumheadlessunity-webgl

Unity3D WebGL Headless not rendering

I have posted the same question over Unity's forum, but there hasn't been any answer and thus posting it here, too.

I have been trying to run a Unity WebGL build in headless mode (through puppeteer) while saving 'screenshots' of the game, but the camera rendering doesn't seem to be working. The resulting images are all black.

It works as expected when not in headless mode (but still WebGL). It also works properly in standalone builds (e.g., windows, mac), through -batchMode.

Here's the code in question:

// Problem seems to be in the following 2 lines
RenderTexture.active = camera.targetTexture;
camera.Render();

// same dimensions, both in headless and not headless
Debug.Log("CAMERA TARGET TEXTURE WIDTH: " + camera.targetTexture.width);
Debug.Log("CAMERA TARGET TEXTURE HEIGHT: " + camera.targetTexture.height);

tempTexture2D = new Texture2D(camera.targetTexture.width, camera.targetTexture.height, TextureFormat.RGB24, false);
tempTexture2D.ReadPixels(new Rect(0, 0, camera.targetTexture.width, camera.targetTexture.height), 0, 0);
tempTexture2D.Apply();

// RGBA(0.000, 0.000, 0.000, 1.000): totally black, when in WebGL headless mode. Works fine otherwise.
Debug.Log(tempTexture2D.GetPixels(100, 100, 1, 1)[0].ToString());

// Encode texture into JPG
byte[] bytes = tempTexture2D.EncodeToJPG();

// byte count is almost half when in headless mode
Debug.Log("IMG " + frameNumber + " byte count: " + bytes.Length);
// save to persistentData (indexedDB in WebGL)
// that data is then read on client side and encoded again

I found some differences between the Webgl headfull and headless versions (respective pictures below). headfull gpu stats headless gpu stats

I have also tried to set --use-gl=swiftshader which has better gpu stats, but still shows everything in black: headless swift shader gpu stats

To be clear, the arguments I'm passing to chromium are the following:

args:[
 '--headless',
 '--hide-scrollbars',
 '--mute-audio',
 '--no-sandbox',
 '--use-gl=swiftshader' // tested with and without
]

The headless log output coming from unity is the following:

PAGE LOG: Loading player data from data.unity3d
PAGE LOG: Initialize engine version: 2018.4.10f1 (a0470569e97b)
PAGE LOG: Creating WebGL 2.0 context.
PAGE LOG: Renderer: WebKit WebGL
PAGE LOG: Vendor: WebKit
PAGE LOG: Version: OpenGL ES 3.0 (WebGL 2.0 (OpenGL ES 3.0 Chromium))
PAGE LOG: GLES: 3
PAGE LOG: EXT_color_buffer_float GL_EXT_color_buffer_float EXT_float_blend GL_EXT_float_blend EXT_texture_filter_anisotropic GL_EXT_texture_filter_anisotropic OES_texture_float_linear GL_OES_texture_float_linear WEBGL_compressed_texture_etc GL_WEBGL_compressed_texture_etc WEBGL_compressed_texture_etc1 GL_WEBGL_compressed_texture_etc1 WEBGL_compressed_texture_s3tc GL_WEBGL_compressed_texture_s3tc WEBGL_debug_renderer_info GL_WEBGL_debug_renderer_info WEBGL_debug_shaders GL_WEBGL_debug_shaders WEBGL_lose_context GL_WEBGL_lose_context
PAGE LOG: OPENGL LOG: Creating OpenGL ES 3.0 graphics device ; Context level <OpenGL ES 3.0> ; Context handle 1
PAGE LOG: UnloadTime: 0.340000 ms
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_OPERATION : glFramebufferTexture2D: <- error from previous GL command
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_OPERATION : GetShaderiv: <- error from previous GL command
PAGE LOG: WebGL: INVALID_OPERATION: renderbufferStorageMultisample: samples out of range
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawElements: framebuffer incomplete
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawArrays: framebuffer incomplete
PAGE LOG: [.WebGL-0x7fcdb69e1600]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glBlitFramebufferCHROMIUM: framebuffer incomplete

Could the problem be the WebGL hardware acceleration altogether? Can I disable it from my WebGL build?

This issue seems to be related with: Rendering WebGL image in headless chrome without a GPU

But that seems to be working properly, at least under MacOS: https://github.com/Apidcloud/WebGLHeadlessRendering

So I'm still assuming it has something to do with Unity3D. Even if headfull, it will become black when using swift shader, with the only gpu stats difference being video decode--hardware acceleration disabled: headfull swift shader gpu stats

Thanks!

Edit with answer (more details on the actual answer below):

It finally works by disabling the anti-aliasing in unity, which seems to throw some OpenGL errors. Works with and without swift shader.

Solution

  • After 2 or 3 days struggling with this and trying to find the exact problem, it finally works by disabling the anti-aliasing in unity, which seems to throw some OpenGL errors.

    That said, the Unity3D WebGL rendering works headlessly (through chromium and puppeteer) even without swift shader, though that is probably what you want for no-gpu scenarios (e.g., some server).

    The log output, though still weird, for both scenarios (with and without swift shader) is the following (note that the framebuffer incomplete errors are gone):

    PAGE LOG: [.WebGL-0x7f842a0f6400]GL ERROR :GL_INVALID_OPERATION : glFramebufferTexture2D: <- error from previous GL command
PAGE LOG: [.WebGL-0x7f842a0f6400]GL ERROR :GL_INVALID_OPERATION : GetShaderiv: <- error from previous GL command

    Thanks!