How can I efficiently send a list of arguments into an Emscripten Web Worker from native JavaScript?
My data (set of arguments) is an inhomogeneous combination of large arrays of Float32Array, UInt16Array, int, etc. The example below works, but I cannot use a list, dictionary, or any other way of making a tuple of TypedArrays mixed with int. Currently, unless the input is one single TypedArray, it is converted into a few bytes. For example, the following line will send three single bytes, one for each argument.
test_data_worker([b8verts, 150000, b8faces]); // doesn't work
My question is in fact, how to encode a tuple into a C struct. I can use libraries such as struct.js but they will be inefficient (needs conversion, and the arguments will not be transferable). What solution Emscripten has for such a case. Note that my special requirement is that I want my JavaScript to postMessage() directly into an Emscripten-compiled C++ function, without being mediated by a sender C++ program on the front side, or a receiver JavaScript code on the Worker side.
<html>
<script>
'use strict';
var worker = new Worker('./emworker.compiled.js');
var api_info = {
test: {
funcName: "worker_function",
callbackId: 3,
},
};
function test_data_worker(arguments_data_buffer) {
// The protocol used by Emscripten
worker.postMessage( {
funcName: api_info.test.funcName,
callbackId: api_info.test.callbackId,
data: arguments_data_buffer,
// 'finalResponse': false,
}
// , [arguments_data_buffer] //uncommet for transferable
);
}
function demo_request() {
var verts = new Float32Array([3.141592651234567890123456780,1,2, 3,4,5, 6,7,8, 9,0.5,1.5]);
var faces = new Uint16Array([0,1,2, 1,2,3, 0,1,3, 0,2,3]);
var b8verts=new Uint8Array(verts.buffer);
var b8faces=new Uint8Array(faces.buffer);
test_data_worker(b8verts); // b8verts.buffer to make it transferrable
// Desired:
//test_data_worker([b8verts, b8faces]); // Doesnt work. sends two bytes instead
//test_data_worker([b8verts, 60, b8faces]); // doesnt work
//test_data_worker({verts:b8verts, d:60, faces:b8faces}); // doesnt work
}
worker.addEventListener('message', function(event) {
// event.data is {callbackId: -1, finalResponse: true, data: 0}
switch (event.data.callbackId) {
case api_info.test.callbackId: //api_revlookup.:
// console.log("Result sent back from web worker", event.data);
// Reinterpret data
var uint8Arr = event.data.data;
var returned_message = String.fromCharCode.apply(null, uint8Arr)
console.log(returned_message);
break;
default:
console.error("Unrecognised message sent back.");
}
}, false);
demo_request();
console.log("request() sent.");
</script>
</html>
and the Worker's C++ code is like
#include <iostream>
#include <emscripten/emscripten.h>
extern "C" {
int worker_function(void* data, int size);
}
std::string hexcode(unsigned char byte) {
const static char codes[] = "0123456789abcdef_";
std::string result = codes[(byte/16) % 16] + ( codes[byte % 16] + std::string());
return std::string(result = codes[(byte/16) % 16] + ( codes[byte % 16] + std::string()));
}
int worker_function(void* data, int size) {
{
std::cout << "As bytes: ";
size_t i = 0 ;
char* char_data = (char*)data;
for (; i < size; ++i) {
std::cout << hexcode((char_data)[i]) << " ";
}
std::cout << std::endl;
}
{
std::cout << "As float: ";
float* typed_data = (float*)data;
size_t typed_size = (size + sizeof(float)-1) / sizeof(float);
for (size_t i = 0; i < typed_size; ++i) {
std::cout << typed_data[i] << " ";
}
std::cout << std::endl;
}
std::string result = std::string("I received ") + std::to_string(size) + " bytes.";
char* resstr = (char*)result.c_str();
emscripten_worker_respond(resstr, result.size()); // not needed really
return 314; // ignored
}
void worker_function2(float*verts, int numverts, int*faces, int numfaces) {
//
}
int main(){return 0;}
which is compiled with Emscripten in the following way:
em++ -s EXPORTED_FUNCTIONS="['_main', '_worker_function' , '_worker_function2' ]" \
-s NO_EXIT_RUNTIME=1 \
-s DEMANGLE_SUPPORT=1 \
-s BUILD_AS_WORKER=1 -DBUILD_AS_WORKER \
-pedantic -std=c++14 \
emworker.cpp \
-o ./emworker.compiled.js
My API on the web worker needs to send and receive tuples of multiple types, i.e. it will have input and output like the following:
typedef struct vf_pair {
std::vector<float> verts, // or a pair<float*,int>
std::vector<int> faces
} mesh_geometry;
int query_shape(char* reduce_operator, char* shape_spec_json, float* points, int point_count);
struct vf_pair get_latest_shape(int obj_id);
struct vf_pair meshify(char* implicit_object);
You can find the patterns I used in the following repo. It is efficient. It exchanges large typed arrays and primitive-typed values back and forth with Web Workers:
https://github.com/sohale/implisolid/blob/master/js_iteration_2/js/implisolid_worker.js