Introducing MagicWorker.js (Apr 13, 2013)

On GitHub: lgarron/MagicWorker.js

You can almost use a .html file like a .app / .exe program… that happens to run in your browser. I wanted to fix the “almost”.

Web Workers

I like making self-contained “web apps”. I was recently porting Josef Jelinek’s ACube program to Javascript for ACube.js. It’s actually very easy to compile C++ to Javascript using Emscripten, and the resulting code runs very fast. However, a fast web app is rather annoying to use if it doesn’t respond while it’s running. Web workers are a great solution to this: Just create a worker in a background thread using

new Worker("worker-file.js");

and communicate by sending JSON messages. Here’s a full working example:

simple.html:

<script>
  var worker = new Worker("simple-worker.js");
  worker.addEventListener("message", function(e) {document.write(e.data)}, false);
  worker.postMessage("Hi!");
</script>

simple-worker.js:

this.addEventListener('message', function(e) {
  this.postMessage("Hello! You sent: " + e.data);
}, false);

The API is pretty simple, but very easy to use and extend. With enough work, this allows you to create HTML files / web apps (with significant computation) that are almost as useful as native applications.

However, there’s a problem: Because of security restrictions, Chrome won’t let you load worker-file.js dynamically if you opened simple.html from your hard drive.

So, I’ve wanted a workaround for this since a while. When I write an HTML app, I don’t always want my users to have to be online to use it. No, the application cache is not a good solution. I want someone be able to download my source code, run it, and be able to edit and test it without having to start a local server. This also makes it easier to test for me, and it’s got that “tinker factor”. I’m a fan of making my work tinkerable, because that’s how I learned to code, and I think it’s the best way to get someone hacking.

First Workaround: Inline Web Workers

It is possible to create a web worker using a different method, if you have the source code of it in a string: this involves creating a blob from the string, giving it a fake URL, and then loading the web worker from that URL. It’s roundabout, but it works on Chrome offline these days. (When I was working on Mark 2 in late 2011, it didn’t.) Here’s how our simple example would work now:

two-sources.html

<script>
  try {
    var worker = new Worker("two-sources-worker.js");
  }
  catch (e) {
    var workerSource = "this.addEventListener(\"message\", function(e) {\n  this.postMessage(\"Hello! You sent: \" + e.data);\n}, false);";
    var blob = new Blob([workerSource]);
    var url = window.URL.createObjectURL(blob);
    var worker = new Worker(url);
  }

  worker.addEventListener("message", function(e) {document.write(e.data)}, false);
  worker.postMessage("Hi!");
</script>

two-sources-worker.js:

this.addEventListener("message", function(e) {
  this.postMessage("Hello! You sent: " + e.data);
}, false);

Unfortunately, this means we have to maintain two copies of the web worker source code. It would be nice if we could load the source of two-sources-worker.js in a way that pleases Chrome, e.g. by adding an extra script tag (or even something like a dummy image tag) and loading its source, but I don’t know of a way. However, the dynamic nature of Javascript has one very useful feature: calling toString() on a function gives us a string of its source code! We can put the source code of the web worker in a function, which we can either create a worker from directly, or use with a script tag to create an inline worker. The code grows once more:

one-source.html

...
    var src = workerCode.toString();
    // Remove the outer "function(){" and "}" parts of the source:
    src = src.slice(0, src.lastIndexOf("}")).substring(src.indexOf("{") + 1);
    var blob = new Blob([src]);
...

one-source-worker.js

var workerCode = function() {
  this.addEventListener("message", function(e) {
    this.postMessage("Hello! You sent: " + e.data);
  }, false);
};

// Detect if we're in a web worker:
if (typeof importScripts === 'function') {
  workerCode();
}

There’s significantly more machinery, but it’s very DRY. Moreover, most of it is the same every time.

MagicWorker.js

That’s where MagicWorker.js comes in. After a lot of experimenting to see what actually worked, I combined all the tricky work into one file. The instructions simplify to this:

  • 1) Add MagicWorker.js to your project.
  • 2) Include MagicWorker.js in your webpage using a normal script tag, followed by each of your web worker files:

    <script src="MagicWorker.js"></script>
    <script src="change-this-to-your-worker-file-name.js"></script>
    
  • 3) Modify each web worker file by prepending the lines:

    (function(f) {if (typeof MagicWorker !== "undefined") {
        MagicWorker.register("change-this-to-your-worker-file-name.js", f);
    } else {f()}})(function() {
    

and appending the line:

    });

(Change "change-this-to-your-worker-file-name.js" to the name of the file.)

Our example in MagicWorker.js:

magical.js:

<script src="MagicWorker.js"></script>
<script src="worker-magical.js"></script>
<script>
  var worker = new Worker("worker-magical.js");
  worker.addEventListener("message", function(e) {document.write(e.data)}, false);
  worker.postMessage("Hi!");
</script>

worker-magical.js:

(function(f) {if (typeof MagicWorker !== "undefined") {
    MagicWorker.register("worker-magical.js", f);
} else {f()}})(function() {

  this.addEventListener("message", function(e) {
    this.postMessage("Hello! You sent: " + e.data);
  }, false);

});

If you remove the MagicWorker.js import, everything still runs fine when you’re not in Chrome offline. That’s the kind of transparency I was aiming for. No need to pre-compile, handle multiple versions of the source, or change a lot of code. Just a drop-in file and a straightforward way to include any web worker file in the process.

The source is on GitHub at lgarron/MagicWorker.js. It also contains all the examples from this post, in case you want to play around with them. There’s also some interesting work to be done. Perhaps someday there will be a nice way to do offline workers in Chrome, but maybe there’s already something more satisfactory right now. Or maybe there are simple tweaks to make this work for shared workers.

I’m still pretty excited, though. .html is the new .exe.

Javascript, Programming, Web No Comments