I have some rust code that compiles to web assembly using wasm-pack and wasm-bindgen. I want to call into this code from a web worklet/worker. The entire app should eventually be just one single *.js file, with everything else inlined.
This is what I imagine my build process to look like:
wasm-pack to compile the rust code to *.wasm and *.js bindings (this step works just fine)webpack to build a self-contained *.js file that I can load as a worklet/worker. The *.wasm must be included in this file. (this step fails)webpack again to build my final app/package, inlining the worklet/worker file from step 2. (this step works just fine)My problem is in step 2: I can't make webpack inline the *.wasm into the worklet/worker file.
I tried this in my webpack config:
entry: {
worker: {
import: './src/worker.ts',
filename: '../lib/worker.js',
}
},
// ...
module: {
rules: [
// ...
{
test: /\.wasm$/,
// 1st option: type: 'webassembly/sync',
// 2nd option: type: 'asset/inline',
},
// ...
],
},
No matter what I do, webpack always emits two files, one worker.js with my worklet/worker script itself, and another one, vendor_my_package_name_wasm_js.js that contains just the *.wasm and its bindings. Obviously, when loading the worker.js as a web worker, it fails - the second file can't be loaded from the worker scope.
My goal is to include everything in worker.js and NOT have a separate file emitted. But how do I do that?
Edit: Documenting steps towards a solution:
Webpack native wasm-loading doesn't seem to allow inlining the wasm file. We can try to use a regular raw-loader:
// in module.rules
{
test: /\.wasm$/,
loader: 'raw-loader',
},
This results in the following error:
ERROR in ./node_modules/my-module/my-wasm-file.wasm
Module parse failed: magic header not detected
File was processed with these loaders:
* ../../node_modules/raw-loader/dist/cjs.js
You may need an additional loader to handle the result of these loaders.
This happens because there's still an implicit default rule that kicks in. We can disable it by overwriting the default rules to only consider json and js files:
// in webpack.config.js
module: {
defaultRules: [
{
type: 'javascript/auto',
resolve: {},
},
{
test: /\.json$/i,
type: 'json',
},
],
rules: [
// ...
{
test: /\.wasm$/,
loader: 'raw-loader',
},
],
},
Now we finally have our worker bundled into a single *.js file! However, when loading it, we end up in this error:
Uncaught ReferenceError: document is not defined
pointing to this piece of webpack-generated code:
/* webpack/runtime/jsonp chunk loading */
/******/ (() => {
/******/ __webpack_require__.b = document.baseURI || self.location.href; // <<< error here
/******/
/******/ // object to store loaded and loading chunks
/******/ // undefined = chunk not loaded, null = chunk preloaded/prefetched
/******/ // [resolve, reject, Promise] = chunk loading, 0 = chunk loaded
/******/ var installedChunks = {
/******/ "myModuleName": 0
/******/ };
/******/
/******/ // no chunk on demand loading
/******/
/******/ // no prefetching
/******/
/******/ // no preloaded
/******/
/******/ // no HMR
/******/
/******/ // no HMR manifest
/******/
/******/ // no on chunks loaded
/******/
/******/ // no jsonp function
/******/ })();
For some reason webpack tries to support loading stuff dynamically (?). We can isolate the problem to this piece of code that was generated by wasm-pack as part of the javascript bindings when using the --target=web CLI argument:
async function init(input) {
if (typeof input === 'undefined') {
input = new URL('my_wasm_file.wasm', import.meta.url);
}
const imports = {};
// ...
Apparently the possibility of having to generate a URL makes webpack rely on document which is not available when loading the worker script in the worker scope. Uncommenting the new URL() part makes the document reference disappear from the webpack output.
Not sure where to go from here. Write my own wasm-loader? I worked on that for a while, base64 encoding the wasm file and inlining it as a string - but then I have to dramatically change the consumer code to manually load the wasm asynchronously. This means that I can't use the wasm-bindgen bindings anymore as they rely on either the URL part shown above (when using --target=web) or the bundling logic of webpack 5 (when using --target=bundler) which I can't get supported from my own simple wasm-loader attempts.
Essentially that means that I have to provide my own JS bindings, which is inconvenient.
There must be a better way - right?
cargo build --target=wasm32/unknown/unknownwasm-bindgen --out-dir=dist --target=web --omit-default-module-path my-wasm-package.wasm.import init, { /* other stuff */ } from 'my-wasm-package';
import wasmData from 'my-wasm-package/my-wasm-package.wasm';
const wasmPromise = init(wasmData);
// ...
module.exports = {
// ...
module: {
rules: [
{
test: /\.ts$/,
loader: 'ts-loader',
options: {
configFile: 'tsconfig.json',
},
},
{
test: /\.wasm$/,
type: "asset/inline",
},
],
},
};
The core problem I observed was that the JS bindings generated by wasm-bindgen contain the URL keyword - which makes webpack initialise in such a way that it expects the document object to be defined. That object is undefined in the worklet scope, so initializing webpack would crash, even if we never entered the code section that contains the URL.
If we don't use wasm-pack to build the wasm and the bindings in one go, we can pass additional arguments to wasm-bindgen - mainly the --omit-default-module-path argument which removes the section with the URL from the bindings. Now webpack won't reference document when it initializes and we can use the bindings without modification.
From here it's simple: We bundle the wasm as a b64 encoded asset and pass it to the init() function that comes with the JS bindings.