To OpenCL from JavaScript via js-ctypes, or, how we rewrote the River Trail Firefox extension

Last September, I began contributing to River Trail, an experimental API for data-parallel programming in JavaScript. It’s been my main project for the last five months, and as of two weeks ago, we announced a new release! This is a post about what I’ve been doing for these five months, and what I hope will happen next with River Trail.

Two interacting components

The River Trail implementation has two interacting components: a JavaScript library, and a Firefox extension. Together, they fight crime provide support for data-parallel programming in JS:

• The River Trail library provides a ParallelArray data structure and a variety of potentially-parallel operations on it, like map, reduce, and filter.

• The River Trail extension enables programs written using the library to actually be run in parallel. This is done using the OpenCL framework, which I’ll say more about in a moment.

I can’t take any credit for the design or implementation of the parallel programming API that the River Trail library provides; all that work had been done by other people before I joined the project. I’ve done some work refreshing the API docs and tutorial, though, and if you’re curious about River Trail’s take on how parallel programming in JavaScript could work, I hope you’ll try out the tutorial and let us know your thoughts.

The part that I can take some credit for is the browser extension, which is what the rest of this post is about.

Background: OpenCL, or, “Why do we need a browser extension?”

The River Trail library offers a parallel programming API that programmers can use, but parallelism is ultimately a matter of how programs are run, not how they’re written. So, if we’re going to say that River Trail is for parallelism, we really ought to make it so that the programs people write using it actually run in parallel somehow, assuming the availability of parallel hardware resources.

Modifying an existing language runtime to make programs written in that language run in parallel can be a daunting task. Certainly this is the case if what we have to modify is a sophisticated, production-quality JS engine like the one in Firefox. So, River Trail doesn’t do that! Instead, we piggyback on the parallelism support that OpenCL provides.

OpenCL is a framework for programming heterogenous parallel hardware. In the OpenCL view of the world, one has access to an assortment of compute devices: CPUs, GPUs, or whatever else happens to be present. The idea is to run so-called kernels on these devices, where a kernel is a program that has been written in OpenCL’s parallelization-friendly flavor of C and then compiled to machine code before being sent to a device to be executed in parallel. You don’t have to worry about writing the kernel in different ways, depending on the device it will eventually run on; the same OpenCL kernel can, at least in principle, run on CPUs, GPUs, FPGAs, or whatever OpenCL-compatible device you have.

To be able to actually run OpenCL kernels, one typically also has to write a so-called “host application” whose job it is to orchestrate interaction with the OpenCL runtime system. The host application has to do things like determine what OpenCL-compatible devices are available, tell kernels to execute on particular devices, and many other housekeeping tasks, like creating an OpenCL “context” in which all these events occur. It does all those things by making calls to the OpenCL C library functions that cause them to happen — or maybe OpenCL C++ wrapper API functions, if you want to get fancy. Those are your options: C or C++.

What the River Trail library does is give you the ability to run JavaScript programs in parallel — leveraging all the OpenCL infrastructure, but without having to write kernels by hand in OpenCL C, and without having to write a host application in C or C++. That having been said, the idea of River Trail is not to convince people that they ought to be programming GPUs with JavaScript! Rather, the idea is: if you have a program, then you ought to be able to run it on the parallel hardware you already have, notwithstanding the fact that the program happens to be written in JavaScript.

So, in River Trail, you can write a JavaScript program that, for instance, creates a ParallelArray and then maps a function f over the array’s elements. The River Trail library will compile the f in question to OpenCL C. It then tells the OpenCL runtime system to further compile the code to a machine-code kernel by invoking the appropriate sequence of incantations from the OpenCL library on your system. OpenCL will also need access to the particular ParallelArray that it’s supposed to be running the kernel on, so, when your code calls the ParallelArray constructor, River Trail tells OpenCL to allocate what’s known as an OpenCL buffer, and it makes sure that the kernel created from f will have access to that buffer. Finally, River Trail tells OpenCL to run the kernel on a device and read back whatever data is in the buffer after the kernel has run.

Again, all this has to happen through calls to the native OpenCL C library. The trouble is that ordinary JavaScript programs running on ordinary web pages can’t call functions in native C or C++ libraries. But browser extensions can! And ordinary JavaScript programs on ordinary web pages can communicate with browser extensions. So, the River Trail Firefox extension provides a bridge by which the River Trail library can talk to your system’s OpenCL. As a result, the River Trail library can run as an ordinary, unprivileged JavaScript program, even though it’s causing native code to be executed.

From XPCOM to pure JavaScript

Browser extensions are typically mostly written in JavaScript. As we’ve just seen, though, the whole point of having a River Trail extension is to provide a way to interact with the OpenCL C library on the user’s system. The traditional approach to interacting with a third-party C library from a Firefox extension has been to create what’s known as a binary XPCOM component, or just “binary component”, which is written in C++ and in XPIDL, Mozilla’s interface description language. This binary component acts as a wrapper for the third-party C library, and it can expose wrapper functions to JavaScript code — both extension code, such as the River Trail extension, and ordinary web code, such as the River Trail library — via the Mozilla XPCOM framework. It’s all an extremely hairy and Mozilla-specific series of tubes, but it works.

The old River Trail implementation used this approach, and for a while, it was sustainable. However, binary components depend on the Mozilla Gecko C++ SDK, which is updated with every release of Firefox, and since Firefox releases happen every six weeks, this meant that the old River Trail extension had to be re-compiled and a new version released every six weeks as well. Unsurprisingly, doing this every six weeks grew tiresome, and so the River Trail team stopped making these updates after Firefox 25, in October 2013. This meant that, starting in December 2013 with Firefox 26, people using a current release of Firefox were not able to run River Trail. When I joined the project in September 2014, it had been nearly a year since there’d been a version of Firefox for which River Trail actually worked.

A final annoyance with binary components is that extensions have to be built separately for each platform on which they must run. The old River Trail extension had to be compiled for both Windows and Mac OS as part of the extension packaging process, and the shipped extension package included two completely separate compiled binaries for Mac and Windows, so the extension package was twice as big as any individual user actually needed it to be. Linux users who wanted to run River Trail had no option but to compile their own extension from source (which involved installing the Gecko SDK, wrangling Makefiles, and so on).

Fortunately, there is a way to avoid all the issues of using binary components to interact with C libraries from JavaScript. These days, Firefox extension developers can use a JavaScript-to-C foreign function interface called js-ctypes. Furthermore, Firefox supports an exportFunction mechanism that allows one to export functions from extension-side JavaScript code so that they can be called from ordinary, user-side JavaScript code.

In the new River Trail extension, we use js-ctypes and exportFunction to completely replace the functionality of the old binary XPCOM component. Because of this, the new extension can be written in 100% JavaScript, and so we can ship a single extension that runs on Windows, Mac, and Linux, and potentially other platforms such as Android. The new, deCOMtaminated extension is just under 1000 lines of JavaScript, as opposed to the old extension, which had around 1500 lines of C++, plus hundreds of lines of header files and XPIDL.

The entire build and release process for the extension is much less painful than it used to be. Linux users no longer have to build the extension from source if they want it in their Firefox; they can just click and install like Mac and Windows users can. Since there is no binary component, the installable package is 25% the size of the previous one for Windows and Mac (25K vs. 101K). And, most importantly, since the extension is no longer tied to a specific Gecko SDK version (and therefore a specific Firefox release) as the previous versions of the extension were, we expect it to be compatible with future Firefox releases. It is already known to work with Firefox 33, 34, and the current release, 35.

From legacy extension to Add-on SDK extension

We also made another change to the River Trail extension that was unrelated to the js-ctypes conversion, but, to me, also made a big difference. The previous River Trail extension was an “overlay”-style extension, which is now also known as a “legacy” extension. With overlay extensions, updating, installing, or disabling an extension requires restarting Firefox.

Overlay extensions are still supported by Firefox, but since Firefox 4, which was released in 2011, it’s been possible to develop “restartless” extensions that can be installed, updated, disabled, and re-enabled without a browser restart. Furthermore, restartless extensions can be developed using the Mozilla Add-on SDK, a set of JavaScript libraries and command-line tools for extension development. So, the new River Trail extension is based on the Add-on SDK.

What’s funny to me now is that for the longest time, I was reluctant to use the SDK. Restartless extensions are, in fact, the only kind of extension that it’s possible to write using the SDK, and, perhaps because of this “limitation” (which is not actually limiting at all), I guess I assumed that the SDK wouldn’t meet our needs. I thought that our extension was surely Too Sophisticated for the SDK, and that, besides, the SDK must not be for Real Programmers; it must be for people who didn’t really know how to write extensions.

As it turns out, I was completely wrong! The SDK is just a really nice collection of APIs for doing useful extension-infrastructure things (some of which I had previously been trying to do myself in various ad-hoc and buggy ways), as well as a handy set of command-line tools for automating certain repetitive aspects of extension development. Basically, the SDK makes it so that we only have to worry about what your extension actually does, instead of all the other scaffolding and boilerplate we had to deal with before. We should have been using it all along.

What about WebCL?

WebCL is an emerging standard that defines bindings to JavaScript from OpenCL. No mainstream web browsers support WebCL, yet (although Intel’s Crosswalk web application runtime for Android does), but if they did, then it would be great for River Trail: rather than having to interact with OpenCL via js-ctypes, we could just call the WebCL JavaScript functions, and we wouldn’t need a browser extension at all.

It’s certainly good that js-ctypes exists; it makes things like a 100% JavaScript implementation of River Trail possible in the Firefox of today. Still, though, having to write code against the OpenCL C API when it’s been shoehorned into JavaScript via js-ctypes is honestly not much fun. Unlike their counterparts in WebCL, the raw OpenCL API functions do very un-JavaScript-y things, such as returning error codes that have to be checked instead of throwing exceptions when something goes wrong. As a result, writing code that makes use of them through js-ctypes doesn’t really feel like writing JavaScript; it feels more like writing C — except without Valgrind or any of the tool support that one would normally hope for when writing C.

But despite WebCL not being well supported yet, the River Trail library can, as of recently, run on top of WebCL. In fact, the library will first look for an available WebCL platform before falling back to using the River Trail extension. This functionality has been tested with Nokia Research’s experimental WebCL Firefox extension, and the River Trail library runs on top of the Nokia WebCL extension on Firefox 33 and 34, with no River Trail extension necessary. (It is currently not working on Firefox 35, not because of any issue with River Trail but apparently due to a known issue with the Nokia extension.)

A personal note

Most of what I’ve written in this post seems pretty obvious to me now, but five months ago, that wasn’t the case at all. When I started working on River Trail, I didn’t know what OpenCL was. I had never written a browser extension. I couldn’t have told you what XPCOM was, other than “uh, some sort of Mozilla thing from the ‘90s?”¹, and I had almost no experience with JavaScript. (I worked at Mozilla for a good chunk of 2011 and 2012, but I was working on Rust, which is kind of its own thing, so I barely touched any of the usual Mozilla infrastructure. I probably spent more time reading MDN in my first day working on River Trail than I did during the entire time I actually worked at Mozilla.)

It took the better part of a month working on River Trail just to get to the point where I could understand the project well enough to know why we needed to rewrite the extension. It took some time, for example, for me to realize that we didn’t want to use js-ctypes to call into the code in the existing extension’s binary component; rather, we wanted to call directly into the OpenCL library itself, and not have to ship a binary component.

Actually becoming conversant in JavaScript also took some time. For a while, I thought that the extension would have to be written in two parts that would communicate asynchronously with each other while somehow presenting a synchronous interface to the rest of the world, and that was quite a deep rabbit hole. And JS as it is written in a Firefox extension differs from how it’s written on a web page. When I found out that I could use modules, for instance, I flipped out and started using them everywhere. Something similar happened with let. You get the idea.

Anyway, I’m happy that I was able to go from knowing nothing about any of this to being a major contributor to River Trail — I’ve learned a lot, and it’s been a lot of fun. Now seems like a good time to mention that we have a couple of open bugs on River Trail (as well as any number of bugs that aren’t known, of course), and so if you’re interested in this project and would like to make a contribution, I’d be happy to try to give you a hand working on these bugs! A lot of people helped me when I was trying to get up to speed with River Trail, and I’d like to pay it forward.

I’m also hoping that the fact that River Trail is open source (under a permissive two-clause BSD-style license) means that people will find interesting ways to use the code for their own purposes. For instance, ShaderDSL.js is a project that uses a hacked version of the River Trail JS-to-OpenCL-C compiler, retargeting it to generate GLSL; I’m sure there are more cool projects that could be done. I’d also really like to see River Trail be a vehicle for academic research, and I’d be happy to help anyone who wants to use it in whatever way that I can.

Thanks to Margaret Staples, Jesse Ruderman, Kirstie Cook, Darius Bacon, Julia Evans, Dan Luu, Sam Tobin-Hochstadt, Claudia Doppioslash, Eric Christopher, and Tatiana Shpeisman for reading drafts of this post.