efficiently painting to a canvas from WASM

Question

I am using the new .NET 8 "WebAssembly Browser App" project template (not Blazor) to try and make a small game engine.

One of the things I need to do is paint to a canvas as fast as possible. I've found a lot of questions asking this specific thing, but they're all a few years old and WASM has changed a lot in that time so I think a lot of the answers are out of date.

This answer in particular is promising, because it implies some sort of memory sharing is possible to avoid copies, but it basically just points to this project. The project is very relevant because it's a game engine in WASM, but unfortunately it's built on Blazor so there seem to be some differences that are preventing it from working for me.

Specifically the code to paint from an IntPtr onto a canvas is

window.PaintCanvas = function PaintCanvas(dataPtr) {
    imageData.data.set(Uint8ClampedArray.from(Module.HEAPU8.subarray(dataPtr, dataPtr + imageData.data.length)));
    context.putImageData(imageData, 0, 0);
    context.drawImage(canvas, 0, 0, canvas.width, canvas.height);
    return true;
}

but I get "Uncaught ReferenceError: Module is not defined" which I'm assuming is because Blazor is doing something to access the WASM heap that the WebAssembly Browser App project is not doing.

So my questions are

1. What's the best way to paint to the canvas as quickly as possible?
2. If it's using Module.HEAPU8 then how do I access that?

EDIT:

OK so I think I've solved the Module.HEAPU8 problem - the scaffolded main.js decomposes the object returned by dotnet.create, but if you don't do that then you get an object with a localHeapViewU8 function which I think is the same as Module.HEAPU8.

So I no longer get a JS error but it also still doesn't work. The IntPtr that I send from C# to JS arrives as undefined, so I think that's the problem, but according to the extremely minimal documentation you should be able to send an IntPtr.

I tried changing it to send a Slice of the array instead of a pinned pointer to the array but it says "The type 'Slice' is not supported by source-generated JavaScript interop. The generated source will not handle marshalling of parameter 'dataPtr'." and it points me to the documentation that says you should be able to send a Slice...

Answer 1

OK I've solved this myself.

First of all I'm not sure this is the most efficient way of painting to a canvas (maybe that's something to do with the graphics card) but it is what I wanted - a way of painting without marshalling all of the pixels and copying them around.

I'm going to put a lot of code in this answer because I wasn't able to find many examples when I was searching for answers, so hopefully this will help someone.

Here is my main.js:

import { dotnet } from './_framework/dotnet.js'

const wasm = await dotnet
    .withDiagnosticTracing(false)
    .withApplicationArgumentsFromQuery()
    .create();

wasm.setModuleImports('main.js', {
    InitCanvas: () => InitCanvas(),
    PaintCanvas: (dataPtr) => PaintCanvas(dataPtr)
});

const width = 320;
const height = 180;

let canvas;
let context;
let imageData;

function InitCanvas() {
    canvas = document.getElementById('screen');
    canvas.width = width;
    canvas.height = height;

    context = canvas.getContext('2d');

    context.mozImageSmoothingEnabled = false;
    context.webkitImageSmoothingEnabled = false;
    context.msImageSmoothingEnabled = false;
    context.imageSmoothingEnabled = false;

    imageData = context.createImageData(width, height);
}

function PaintCanvas(dataPtr) {
    window.requestAnimationFrame(function() {
        const slice = wasm.localHeapViewU8().subarray(dataPtr, dataPtr + imageData.data.length);
        const bytes = Uint8ClampedArray.from(slice);

        imageData.data.set(bytes);
    
        context.putImageData(imageData, 0, 0);

        context.drawImage(canvas, 0, 0, canvas.width, canvas.height);
    });
}

const config = wasm.getConfig();
const exports = await wasm.getAssemblyExports(config.mainAssemblyName);

await dotnet.run();

exports.Tactics.Run();

You can see (as I mentioned in the edit on the question) how I solved the Module.HEAPU8 problem - I'm storing the object returned by dotnet.create in a constant called wasm so that I can later call wasm.localHeapViewU8() to get a reference to the WASM heap.

Then the problem was that the IntPtr that I sent from C# to JS was always undefined. That turned out to be a mistake in my code - in the wasm.setModuleImports call I wasn't passing the dataPtr parameter through to window.PaintCanvas.

Finally I couldn't tell that it was working because the test pixels I was sending over were white and transparent, so they weren't visible on the white background 🤦‍♂️

Here is the C# side in case it's useful to anyone:

using System;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.JavaScript;

public partial class Tactics {
    readonly static uint[] _screen = new uint[320 * 180];

    [JSExport]
    internal static void Run() {
        // set pixels in _screen here - each entry in the array represents one pixel so you're going to need to convert the r/g/b/a of your pixel to a uint

        var gchscreen = GCHandle.Alloc(_screen, GCHandleType.Pinned);

        try {
            var pinnedscreen = gchscreen.AddrOfPinnedObject();

            PaintCanvas(pinnedscreen);
        } finally {
            gchscreen.Free();
        }
    }

    [JSImport("InitCanvas", "main.js")]
    internal static partial void InitCanvas();

    [JSImport("PaintCanvas", "main.js")]
    internal static partial void PaintCanvas(IntPtr dataPtr);
}