I'm currently writing on a program, where I have to deal with huge arrays. I can however split those arrays. My plan now is, to process the arrays in different web workers. I have however never worked with them and do have several questions:

1. How would I run several web workers? I tried a for-loop looking like that:

for(i = 0; i < eD.threads; i++){
    //start workers here 
    var worker = new Worker("js/worker/imageValues.js");
    worker.postMessage(brightness, cD.pixels[i]);
}

Here I do get the error, that the object couldn't be cloned. Which seems logical. I guess it would be better to save them in an Array?

2. How would I control that all have finished their work? (I need to reassembly the array and work with it later)

3. How many web workers really bring an improvement?

4. Is there any advanced tutorial, besides the MDN-entry?

Thank you!

I'm currently writing on a program, where I have to deal with huge arrays. I can however split those arrays. My plan now is, to process the arrays in different web workers. I have however never worked with them and do have several questions:

1. How would I run several web workers? I tried a for-loop looking like that:

for(i = 0; i < eD.threads; i++){
    //start workers here 
    var worker = new Worker("js/worker/imageValues.js");
    worker.postMessage(brightness, cD.pixels[i]);
}

Here I do get the error, that the object couldn't be cloned. Which seems logical. I guess it would be better to save them in an Array?

2. How would I control that all have finished their work? (I need to reassembly the array and work with it later)

3. How many web workers really bring an improvement?

4. Is there any advanced tutorial, besides the MDN-entry?

Thank you!

• javascript
• arrays
• html
• performance
• web-worker

• For #2: You'd have to use postMessage from the Worker to send back a message to this parent JS saying that worker is done processing (and probably pass the results)...so you'd have to add an onmessage listener for each Worker, in the loop. You'd store all these responses in probably an array. When the number of responses in the array is the same as the number of workers created, that should mean all are done...and you can deal with that array you created. This could be made "cleaner" by using the concept of jQuery's deferred objects – Ian Commented Aug 12, 2013 at 16:20
• 2 "Cannot be cloned" here means that what you're passing to the worker couldn't be cloned. To avoid race conditions, objects are never shared between workers and the main thread. Objects must either be copied into the worker's memory space, or (with Blobs and typed arrays) the main thread may relinquish memory space to a worker and lose the reference to the object. – apsillers Commented Aug 12, 2013 at 16:53
• Ok. Read about it, but forgot it. Anyway. If I create a temporary variable in the loop, safe the array as a string and pass that string on, would that be a better solution? Somewhere I read, that I only send Strings to workers, somewhere else it says, that I can transfer variables? Thank you both for your explanation though, helps me to get started! – stiller_leser Commented Aug 12, 2013 at 16:59
• 1 @stiller_leser According to the spec, worker's postMessage can pass virtually anything except functions and host objects (e.g. DOM nodes). If you try to pass an object or array that contains functions or DOM nodes, the copy will fail. – apsillers Commented Aug 12, 2013 at 17:07

2 Answers 2

1. How would I run several web workers? I tried a for-loop looking like that:

There's no problem with creating more than one worker, even if you don't keep track of them in an array. See below.

2. How would I control that all have finished their work? (I need to reassembly the array and work with it later)

They can post a message back to you when they're done, with the results. Example below.

3. How many web workers really bring an improvement?

How long is a piece of string? :-) The answer will depend on the target machine on which this is running, the nature of the code running and what other resources it uses, etc. You'll have to tune for your target environment and task. You can find out how many cores are available using navigator.hardwareConcurrency on browsers and Deno or os.cpus on Node.js, but remember you're sharing those cores with other processes on the machine, it's not just your JavaScript code that's running.

4. Is there any advanced tutorial, besides the MDN-entry?

There isn't a lot "advanced" about web workers. :-) I found this article was sufficient.

Here's an example running five workers and watching for them to be done:

Main window:

display("Starting workers...");
let running = 0;
for (let n = 0; n < 5; ++n) {
    workers = new Worker("worker.js");
    workers.onmessage = workerDone;
    workers.postMessage({ id: n, count: 10000 });
    ++running;
}
function workerDone(e) {
    --running;
    display("Worker " + e.data.id + " is done, result: " + e.data.sum);
    if (running === 0) {
        // <== There is no race condition here, see below
        display("All workers complete");
    }
}
function display(msg) {
    const p = document.createElement("p");
    p.innerHTML = String(msg);
    document.body.appendChild(p);
}

worker.js:

this.onmessage = (e) => {
    let sum = 0;
    for (let n = 0; n < e.data.count; ++n) {
        sum += n;
    }
    this.postMessage({ id: e.data.id, sum: sum });
};

About the race condition that doesn't exist: If you think in terms of true pre-emptive threading, then you might think: I could create a worker, increment running to 1, and then before I create the next worker I could get the message from the first one that it's done, decrement running to 0, and mistakenly think all the workers were done.

That can't happen in the environment web workers work in. Although the environment is welcome to start the worker as soon as it likes, and a worker could well finish before the code starting the workers finished, all that would do is queue a call to the workerDone function for the main JavaScript thread. There is no pre-empting. And so we know that all workers have been started before the first call to workerDone is actually executed. Thus, when running is 0, we know they're all finished.

Final note: In the above, I'm using onmessage = ... to hook up event handlers. Naturally that means I can only have one event handler on the object I'm doing that with. If you need to have multiple handlers for the message event, use addEventListener. All browsers that support web workers support addEventListener on them (you don't have to worry about the IE attachEvent thing).

stiller_leser. If you still work on the problem, take a look at the ng-vkthread plugin. It allows you to implement parallel computing technique, which you're trying to develop in your project. With this plugin you can execute multiple functions in multiple threads simultaneously.

Function can be defined directly in the main thread or called from an external javascript file.

Function can be:

• Regular functions
• Object's methods
• Functions with dependencies
• Functions with context
• Anonymous functions

Basic usage:

/* function to execute in a thread */
function foo(str) {
    return str.toUpper();
}

function bar(str) {
    return str.toUpper();
}

// to execute these 2 functions in 2 threads: //

/* create objects, which you pass to vkThread as an argument*/
var param1 = {
      fn: foo,
      args: ['hello']
  };
var param2 = {
      fn: bar,
      args: ['world']
  }

/* run thread */
vkThread.execAll([param1,param2]).then(
  function (data) {
    $scope.repos = data[0] + ' ' + data[1]; // <-- HELLO WORLD
  }
);

Examples and API doc: http://www.eslinstructor.net/ng-vkthread/demo/

Hope this helps,

--Vadim