Oink-based Patch Generation

For Mozilla 2, I proposed that we use Oink, or really Elsa, to build a tool that can help automate deCOMtamination, switching to C++ exceptions instead of nsresults, and similar tasks beyond the reach of sed or perl. The idea is to specify pattern and replacement abstract syntax trees (ASTs), and have the tool match the pattern against sub-tree instances, producing a version of the source with the replacement AST pretty-printed instead of the matching sub-tree.

Graydon came up with the insight of using source coordinates, which the Oink framework tracks accurately, to tell which lines in a given primary source (.cpp file, not .i file that results from the C pre-processor) correspond to a particular AST. The tool would read the source, counting lines until the range to delete is reached, then copy each such line matching the pattern to replace with a “- ” prepended. Then write lines pretty-printed from the replacement AST with “+ ” prepended. Add a bit more boilerplate and context lines, and you have a patch.

Graydon didn’t just make this breakthrough, he prototyped it, with a hardwired pattern to replace struct foo pointers with struct bar pointers in a test file:

$ cat test.cc
struct foo {};
struct holdsAFoo
{
char x;
static const foo * const y;
int *z;
};
$ ./staticprint -fs-print-rewrite test.cc
--- test.cc
+++ test.cc
@@ -5,1 +5,1 @@
-  static const foo * const y;
+static struct bar const * const y;
$ ./staticprint -fs-print-rewrite test.cc | patch -p0
patching file test.cc
$ cat test.cc
struct foo {};
struct holdsAFoo
{
char x;
static struct bar const * const y;
int *z;
};

All this wants is an Oink tool of its own (instead of a hack to staticprint), a little more indentation fidelity (bonus points for respecting the source file’s Emacs and vim modelines), C++ purity (no struct before bar needed), patch context and hunk coalescing, and generalized AST pattern/replacement options, and we’ll be able to make major systematic changes by generating patches. Neat!

/be

Project Tamarin

I just committed the initial revision of mozilla/js/tamarin, the open-source ActionScript Virtual Machine contributed by Adobe. But I had nothing to do with this fine work. It’s the product of some talented folks at Adobe (originally at Macromedia; at least one of whom has moved on and is doing a startup).

Tamarin, the open-source AVM, is the JIT-oriented JavaScript VM that I mentioned in my last post. We have already developed, via a two-day marathon hacking session, a set of proof-of-concept patches to integrate it with SpiderMonkey. The resulting Franken-VM was compiling and linking when I last laid hands on it, but not quite limping. This exercise produced many insights to guide more durable integration. And it was great fun interacting with the new open-source contributors from Adobe, whom I would like to welcome to Mozilla:

  • Jeff Dyer, self-hosting compiler architect
  • Steven Johnson, Tamarin developer
  • Tom Reilly, garbage collector developer
  • Rick Reitmaier, JIT developer
  • Dan Smith, Tamarin module owner
  • Edwin Smith, Tamarin creator and VM architect
  • Erik Tierney, Tamarin developer

My apologies to those whose names I haven’t yet learned. I’ve had the pleasure of getting to know most of the above-named folks through the ECMA TG1 (JavaScript standards) group. They are good people and skillful hackers, and I’m grateful that they are adding their strength to the Mozilla project.

I should point out that, although Adobe did not release open source for its existing, free AS3 compiler, it did contribute a self-hosting (AS3-in-AS3) compiler prototype written by Jeff Dyer. We intend to develop this jointly into an ES4/JS2 self-hosting compiler, implementing the full and final ECMA-262 Edition 4 spec when it is done. In the mean time, if you want to generate ABC for Tamarin, you have open source options.

I’ll be on IRC in a little over nine hours to chat about this great news and answer questions, and then I’m off to Web 2.0. Later this week I will blog again about some early progress on another Mozilla 2 front.

/be

Update: Frank Hecker of the Mozilla Foundation clearly and concisely answers many basic questions about What This All Means, questions to which I tend to give short shrift in this too-technical blog of mine. Recommended.