I want to ask if it's faster and more memory efficient to draw a series of images into a canvas using WebAssembly/WASM?

I'm asking this since I have tested to draw series of images (animation) both in the main UI and with web workers. For few canvases (1-5 canvases) the performance is bearable, however for more canvases (like 20-25) the scenario gets a little different, rendering bees slow motion.

Here's the code in main UI:

const videoDecoder = new Worker("videoDecoder.js");
const canvas = document.querySelector("#canvas");
const offscreen = canvas.transferControlToOffscreen();
videoDecoder.postMessage({ action: 'init', canvas: offscreen }, [offscreen] );

Here's my web worker:

onmessage = async function (e) {
    const blob = e.data;
    blob.arrayBuffer().then(arrayBuffer => {
        const uint8Array = new Uint8Array(arrayBuffer);
        for(;;;) {
           const offsetIdx = ...;
           const endIdx = ...;
           const jpegArray = uint8Array.slice(offsetIdx, endIdx);
           const blob = new Blob([jpegArray], {type: "image/jpeg"});
           drawImage(blob);
        }
    }
}

async function drawImage(blob) {
    const bmp = await createImageBitmap(blob);
    context.drawImage(bmp, 0, 0);
    bmp.close();
}

This works fine it can render the images to the browser from web worker, but as mentioned, at 1-5 canvases it is still smooth, but with more canvases the rendering in the browser gets very slow.

So I want to ask is it faster to draw a series images on a Canvas with WebAssembly? Or there will be no theoretical performance gain pared to doing it now with the web worker approach?

I want to ask if it's faster and more memory efficient to draw a series of images into a canvas using WebAssembly/WASM?

I'm asking this since I have tested to draw series of images (animation) both in the main UI and with web workers. For few canvases (1-5 canvases) the performance is bearable, however for more canvases (like 20-25) the scenario gets a little different, rendering bees slow motion.

Here's the code in main UI:

const videoDecoder = new Worker("videoDecoder.js");
const canvas = document.querySelector("#canvas");
const offscreen = canvas.transferControlToOffscreen();
videoDecoder.postMessage({ action: 'init', canvas: offscreen }, [offscreen] );

Here's my web worker:

onmessage = async function (e) {
    const blob = e.data;
    blob.arrayBuffer().then(arrayBuffer => {
        const uint8Array = new Uint8Array(arrayBuffer);
        for(;;;) {
           const offsetIdx = ...;
           const endIdx = ...;
           const jpegArray = uint8Array.slice(offsetIdx, endIdx);
           const blob = new Blob([jpegArray], {type: "image/jpeg"});
           drawImage(blob);
        }
    }
}

async function drawImage(blob) {
    const bmp = await createImageBitmap(blob);
    context.drawImage(bmp, 0, 0);
    bmp.close();
}

This works fine it can render the images to the browser from web worker, but as mentioned, at 1-5 canvases it is still smooth, but with more canvases the rendering in the browser gets very slow.

So I want to ask is it faster to draw a series images on a Canvas with WebAssembly? Or there will be no theoretical performance gain pared to doing it now with the web worker approach?

• javascript
• html
• html5-canvas
• webassembly
• emscripten

• webassembly has no access to DOM - a canvas is part of the DOM - so, no, it would not be possible – Jaromanda X Commented Apr 22, 2020 at 22:53
• @jaromanda-x According to this stackoverflow./questions/42806037/modify-canvas-from-wasm and other blog articles I found online, its possible to "draw" to the canvas from WASM using shared memory. – Questionaut Commented Apr 22, 2020 at 22:57
• well, try it and see for yourself – Jaromanda X Commented Apr 22, 2020 at 22:59
• My concern is mainly in the performance aspect which before I try to "rewrite" my web worker into C++ for wasm I want to ask if anyone have experience how much performance gain vs doing it on Javascript alone--or there is none so I would like to know upfront if there's a point on taking up the task on doing WASM at all. – Questionaut Commented Apr 22, 2020 at 23:06
• 1 Here's a demo: hellorust./demos/canvas/index.html – Questionaut Commented Apr 22, 2020 at 23:24

2 Answers 2

WebAssembly may be faster for plex putation on a byte array, but the main JavaScript UI thread will still be doing the rendering of the ArrayBuffer to canvas. As long as you're "reusing" ArrayBuffer instances by passing objects with the Transferable interface between main UI thread and WebWorker, you should be able to easily update the canvas performantly.

When rendering updates to multiple canvas objects, consider batching them within a requestAnimationFrame() function to minimize repainting of the pixels in the browser. This should significantly improve browser frame rate performance.

So I want to ask is it faster to draw a series images on a Canvas with WebAssembly?

WebAssembly can be faster than JavaScript, but it isn't always. It very much depends on the type of putation you are performing. Probably the greatest strength of WebAssembly is that the performance is predictable and constant. This blog post gives probably the most informative real-world performance results:

https://hacks.mozilla/2018/01/oxidizing-source-maps-with-rust-and-webassembly/

Or there will be no theoretical performance gain pared to doing it now with the web worker approach?

The two methods are in no way mutually exclusive! You can run WebAssembly within a Web Worker and use shared memory (similar to a SharedArrayBuffer) to allow cross-thread access of your image data. This blog post gives an example of how WebAssembly can be used to pute an algorithm across multiple worker threads:

https://blog.scottlogic./2019/07/15/multithreaded-webassembly.html

In response to your specifics, a video decoders (which are typically written in C++) are a very good candidate for WebAssembly. I'd suggest creating multiple web workers, each with a WebAssembly-powered decoder.