Search code examples
javascriptc++web-workeremscripten

Emscripten webworker - Own messages and dependencies

I have a C++ project that I compile to Javascript using emscripten. This works, however, for resource limits and interactivity reasons I would like to run this inside a webworker.

However, my project uses the stdin. I found a way to provide my own implementation of stdin by overwriting Module['stdin'] with a function that returns a single character at a time of the total stdin, and closes with 0 as EOF. This works when the script runs inside the page, as the Module object present in the html file is shared with the script.

When you run as a webworker though, this module object is not shared. Instead, message passing makes sure the regular functionality of Module still works. This does not include 'stdin'.

I worked around this by modifying the output javascript:

  • A: Adding an implementation of a Module object that includes this stdin specification. This function is modified to read a variable of the webworker as if it were the stdin and feed this on a per-character basis.
  • B: Changing the onmessage of the webworker to call an additional function handling my own events.
  • C: This additional function listens to the events and reacts when the event is the content of stdin, by setting the variable that the stdin function I specified reads.
  • D: adding and removing run dependencies on this additional event to prevent the c++ code running without the stdin specified.

In code:

Module['stdin_pointer'] = 0;
Module['stdin_content'] = "";

Module['stdin']=(function () {
   if (Module['stdin_pointer'] < Module['stdin_content'].length) {
      code = Module['stdin_content'].charCodeAt(Module['stdin_pointer']);
      Module['stdin_pointer']=Module['stdin_pointer']+1;
      return code;
   } else {
      return null;
   }
});

external = function(message){
   switch(message.data.target){
      case 'stdin' : {
          Module['idpCode'] = message.data.content;
          removeRunDependency('stdin');
          break;                        
      }
      default: throw 'wha? ' + message.data.target;
   }
};

[...]

addRunDependency("stdin");

[...]
//Change this in the original onmessage function:
//      default: throw 'wha? ' + message.data.target;
//to
default: {external(message);}

Clearly, this a & c part is quite easy because it can be added at the start (or near the start) of the js file, but b & d (adding your own dependencies and getting your own messagehandler in the loop) requires you to edit the code inline. As my project is very large, finding the necessary lines to edit can be very cumbersome, only more so in optimized and mimified emscripten code. Automatic scripts to do this, as well as the workaround itself, are likely to break on new emscripten releases.

Is there a nicer, more proper way to reach the same behavior?

Thank you!

//EDIT: The --separate-asm flag is quite helpful, in the respect that the file that I must edit is now only a few lines long (in mimified form). It greatly reduces the burden, but it is still not a proper way, so I'm reluctant to mark this as resolved.

Solution

  • The only way I know of achieving what you want is to not use the Emscripten-supplied worker API, and roll your own. All the details are probably beyond the scope of a single question, but at a high level you'll need to...

    As a side-note, I have found that the Emscripten-supplied C++ wrappers for JavaScript functionality, such as workers, graphics, audio, http requests etc, are good to get going at first, but have limitations and don't expose everything that is technically possible. I have often had to roll my own to get the functionally needed. Although not for the same reasons, I have also had to write my own API for workers.