Introducing MagicWorker.js (Apr 13, 2013)

On GitHub: lgarron/MagicWorker.js

You can almost use a .html file like a .app / .exe program… that happens to run in your browser. I wanted to fix the “almost”.

Web Workers

I like making self-contained “web apps”. I was recently porting Josef Jelinek’s ACube program to Javascript for ACube.js. It’s actually very easy to compile C++ to Javascript using Emscripten, and the resulting code runs very fast. However, a fast web app is rather annoying to use if it doesn’t respond while it’s running. Web workers are a great solution to this: Just create a worker in a background thread using

new Worker("worker-file.js");

and communicate by sending JSON messages. Here’s a full working example:

simple.html:

<script>
  var worker = new Worker("simple-worker.js");
  worker.addEventListener("message", function(e) {document.write(e.data)}, false);
  worker.postMessage("Hi!");
</script>

simple-worker.js:

this.addEventListener('message', function(e) {
  this.postMessage("Hello! You sent: " + e.data);
}, false);

The API is pretty simple, but very easy to use and extend. With enough work, this allows you to create HTML files / web apps (with significant computation) that are almost as useful as native applications.

However, there’s a problem: Because of security restrictions, Chrome won’t let you load worker-file.js dynamically if you opened simple.html from your hard drive.

So, I’ve wanted a workaround for this since a while. When I write an HTML app, I don’t always want my users to have to be online to use it. No, the application cache is not a good solution. I want someone be able to download my source code, run it, and be able to edit and test it without having to start a local server. This also makes it easier to test for me, and it’s got that “tinker factor”. I’m a fan of making my work tinkerable, because that’s how I learned to code, and I think it’s the best way to get someone hacking.

First Workaround: Inline Web Workers

It is possible to create a web worker using a different method, if you have the source code of it in a string: this involves creating a blob from the string, giving it a fake URL, and then loading the web worker from that URL. It’s roundabout, but it works on Chrome offline these days. (When I was working on Mark 2 in late 2011, it didn’t.) Here’s how our simple example would work now:

two-sources.html

<script>
  try {
    var worker = new Worker("two-sources-worker.js");
  }
  catch (e) {
    var workerSource = "this.addEventListener(\"message\", function(e) {\n  this.postMessage(\"Hello! You sent: \" + e.data);\n}, false);";
    var blob = new Blob([workerSource]);
    var url = window.URL.createObjectURL(blob);
    var worker = new Worker(url);
  }

  worker.addEventListener("message", function(e) {document.write(e.data)}, false);
  worker.postMessage("Hi!");
</script>

two-sources-worker.js:

this.addEventListener("message", function(e) {
  this.postMessage("Hello! You sent: " + e.data);
}, false);

Unfortunately, this means we have to maintain two copies of the web worker source code. It would be nice if we could load the source of two-sources-worker.js in a way that pleases Chrome, e.g. by adding an extra script tag (or even something like a dummy image tag) and loading its source, but I don’t know of a way. However, the dynamic nature of Javascript has one very useful feature: calling toString() on a function gives us a string of its source code! We can put the source code of the web worker in a function, which we can either create a worker from directly, or use with a script tag to create an inline worker. The code grows once more:

one-source.html

...
    var src = workerCode.toString();
    // Remove the outer "function(){" and "}" parts of the source:
    src = src.slice(0, src.lastIndexOf("}")).substring(src.indexOf("{") + 1);
    var blob = new Blob([src]);
...

one-source-worker.js

var workerCode = function() {
  this.addEventListener("message", function(e) {
    this.postMessage("Hello! You sent: " + e.data);
  }, false);
};

// Detect if we're in a web worker:
if (typeof importScripts === 'function') {
  workerCode();
}

There’s significantly more machinery, but it’s very DRY. Moreover, most of it is the same every time.

MagicWorker.js

That’s where MagicWorker.js comes in. After a lot of experimenting to see what actually worked, I combined all the tricky work into one file. The instructions simplify to this:

  • 1) Add MagicWorker.js to your project.
  • 2) Include MagicWorker.js in your webpage using a normal script tag, followed by each of your web worker files:

    <script src="MagicWorker.js"></script>
    <script src="change-this-to-your-worker-file-name.js"></script>
    
  • 3) Modify each web worker file by prepending the lines:

    (function(f) {if (typeof MagicWorker !== "undefined") {
        MagicWorker.register("change-this-to-your-worker-file-name.js", f);
    } else {f()}})(function() {
    

and appending the line:

    });

(Change "change-this-to-your-worker-file-name.js" to the name of the file.)

Our example in MagicWorker.js:

magical.js:

<script src="MagicWorker.js"></script>
<script src="worker-magical.js"></script>
<script>
  var worker = new Worker("worker-magical.js");
  worker.addEventListener("message", function(e) {document.write(e.data)}, false);
  worker.postMessage("Hi!");
</script>

worker-magical.js:

(function(f) {if (typeof MagicWorker !== "undefined") {
    MagicWorker.register("worker-magical.js", f);
} else {f()}})(function() {

  this.addEventListener("message", function(e) {
    this.postMessage("Hello! You sent: " + e.data);
  }, false);

});

If you remove the MagicWorker.js import, everything still runs fine when you’re not in Chrome offline. That’s the kind of transparency I was aiming for. No need to pre-compile, handle multiple versions of the source, or change a lot of code. Just a drop-in file and a straightforward way to include any web worker file in the process.

The source is on GitHub at lgarron/MagicWorker.js. It also contains all the examples from this post, in case you want to play around with them. There’s also some interesting work to be done. Perhaps someday there will be a nice way to do offline workers in Chrome, but maybe there’s already something more satisfactory right now. Or maybe there are simple tweaks to make this work for shared workers.

I’m still pretty excited, though. .html is the new .exe.

Javascript, Programming, Web No Comments

Cube Flag (Jul 04, 2007)

Cube Flag

Since I live in America, I suppose I should be patriotic… Well, I’ll use this opportunity to show off my cool Cube Flag! (Huge – 6000×4500)

Here’s an animation.
This is based on (actually, taken from) my animated US Open logo. Made with POV-Ray, of course. (Source code will be up in a few years. :-)

Cubing, Old Updates, POV-Ray No Comments

Waltzing Cube and Alg Display (Jun 26, 2007)

A lot of updates are missing here…

Anyhow, I created a synchronized “Waltzing Cube” animation.
I also implemented an alg display feature (actually, two) as a Firefox search engine.

Algorithms, Cubing, Old Updates, Programming No Comments

GEB Blocks (Jan 12, 2007)

I got this idea from GEB (Gödel, Escher, Bach). Find your initials in the animation.

LG Initials

LG Initials

Math, POV-Ray No Comments

Alg of the Week (Jan 01, 2007)

This year, I am introducing the Alg of the Week. The first alg is cube-in-cube-in-cube.

Algorithms, Cubing, Old Updates, Programming No Comments

Countdown Animation (Dec 31, 2006)

Happy New Year!
The POV-Ray source code; directions for custom render:
Boot POV-Ray; render 345 frames at good resolution. Align music beat every second (starting 0.5 seconds). Concatenate and merge at 30 FPS; enjoy any time.

60-second version.

Old Updates, POV-Ray No Comments

POV-Ray Short Code Contest Entries (Dec 24, 2006)

Did I ever mention my two entries into the POV-Ray Short Code Contest 4? Guess which two are mine.

Old Updates, POV-Ray No Comments

Rubik’s Cube Animation System (Nov 26, 2006)

November 26: I finally cleaned up POV-Ray my Rubik’s cube animation system; it’s not efficient, but it’s a start… The file as is rendered at high resolution should give a nice image. Use +kff200 for a joined T and J PLL animation.

You can get POV-Ray here. It’s a great program (although raytracing animation is far from real-time, so I’m going to try to port the code to OpenGL).

December 6: The Rubik’s cube animation system is now capable of parsing an algorithm as a string in the familiar URFLBD format (with 2, ‘, 2′[='2] suffixes). I could add slice turns and double-layer twists, if requested…

Cubing, Old Updates, POV-Ray No Comments

Potential Square-1 Shape Animations (Nov 02, 2006)

Michael Fung gave me permission to render his Square-1 shape algorithms for getting into cube shape. I’ll have to figure out a way to put up the animations, but for now, a PDF. I’ll investigate the case of several missing images.

Cubing, Old Updates, POV-Ray No Comments

Twisty Pumpkin (Oct 31, 2006)

A Twisty Pumpkin.

Cubing, Old Updates, POV-Ray No Comments