ES4 News and Opinion

ES4 News

Some news on language size: the proposed ECMAScript 4th edition (ES4) grammar is a bit more than twice as big as ES3’s, counting several ways (concrete productions, abstract syntax tree node types):

Syntax                   ES3   ES4
------                   ---   ---
Concrete                 238   490
Type Exprs                    50
Classes & Interfaces          23
Patterns                      17
Abstract                  33    77

This is not out of line given the eight years since ES3, which came less than three years after ES1. Much of ES4 has been developed and shipped in ES3-based languages: AS3 in Flash, JS1.6 and 1.7 in Firefox, Opera’s extensions including array destructuring. And we’re being generous with syntactic conveniences, which desugar to a smaller core language. On the compatibility front, we have a new document on known incompatibilities between ES3 (ECMA-262 Edition 3 the spec, not real-world behavior of ES3 implementations in browsers — note well) and ES4.


Chris Wilson: “… I don’t think the other Microsoft guys ever intended to say ‘everything about ES4 is bad’.” Pratap Lakshman (Microsoft): “We do not support or agree to the current ES4 proposal, either in whole or in part.” Individual intentions can be hard to judge, and anyway matter less than effects. My point is not to guess about intentions, only to point to statements and actions.


It’s a relief to learn that Chris’s charge of “shouting down dissent” — confusingly juxtaposed with his words “I think it’s a shame that dissenting opinion has been hidden from view”, which seem to point to the ECMAScript standards body’s majority in a way that left many readers thinking that someone in Ecma TC39-TG1 actually shouted at Microsoft reps during a meeting — was really just a complaint about bloggers “shouting” (which is news-at-11 fodder, don’t you think? I’ve seen shouting from all sides on this one).

That leaves the charge of “ignoring”, which was not aimed at the blogosphere — that accusation was aimed straight at “the TG-1 committee chair”, presumably me (I’m TG1 Convenor). But it is simply false to claim I’ve ignored compatibility concerns or anything else raised by a member of TG1.
Still, if we can make progress by agreeing that bloggers sometimes shout, or show bad manners, I’m all for it. My manners aren’t perfect, and I’m sorry when they fail. But morals trump manners, and the overarching issue before us remains: is it right for ES4 to be stalled in committee, or for ES3 to be forked into a now-hidden “ES3.1”, because Microsoft and Doug Crockford object to ES4 on general grounds?


Whatever you think of ES4 (and it’s not done yet — we just released a reference implementation update), the history nicely reconstructed by Neil Mix should make clear that it has been a long time coming, most of the time without anyone refusing to support it “in whole or in part”. The July 2006 meeting notes include a statement from Pratap that ES4 would be implemented in the next major IE release after IE7.

Of course this all changed, starting early in 2007, officially only in March. But why did it change? Was all the pre-history, including Microsoft’s implementation of JScript.NET, a big mistake? Did JScript.NET really “fail”, and fail in any way that bears on ES4 itself or its future on the client side of the web? I think not, and the generalities about breaking the web, or the web not needing to change much, do not explain this about-face from 2006 to 2007.

Anyone who reads the face-to-face meeting notes can see that everyone working on ES4, including me, would work on topics of interest with whomever attended meetings. And we worked together well enough when focused on technical issues. Allen Wirfs-Brock of Microsoft, in particular, is very knowledgeable and insightful.


But the cold hard fact, documented in that September face-to-face meeting note, is that the dissenting minority in TG1 rejects ES4 in whole and in part. There does not seem to be room for compromise.
As TG1 Convenor, I’ve tried to keep any “3.1” work in the context of ES4, to avoid a fork of ES3 into incompatible derivative languages. In particular, any ES3.1 should be a subset of the ES4 reference implementation, so it can be tested. This seemed agreeable in March, and Allen reiterated agreement in April.
Since April, it has been impossible for the majority of TG1 to evaluate ES3.1. There has been no work on the wiki, and almost no communication to the majority of TG1 about what’s in ES3.1 or how it will be tested for backward compatibilty with ES3 and forward compatibility with ES4.

Do The Right Thing

Neither a hidden and unready alternative proposal, nor general objections from a minority in the group, should halt a standard in progress. ES4 should be delayed or abandoned only on technical demerits, demonstrated specifically. There are enough implementors working together to make a standard, and ES4 as it has evolved has always been the expected successor standard. To block it based on general fears and opinions prejudges both the final standard and the market for it. The market, not Ecma TC39-TG1, should decide ES4’s fate.

It should be clear by now that the majority of TG1, who favor ES4, want to keep the group from splitting, or otherwise risking incompatible forks of ES3. Everyone loses in that scenario. We wouldn’t have spent all the time in the wiki and at meetings if we all, including me, didn’t want a win-win outcome for all players. And I personally suggested to Chris in March that Microsoft could be a big winner by backing ES4 and supporting it in IE.

Is it still possible for everyone to win? I have to say I’m not as positive as I used to be. If we all work on testable compatibility via the reference implementation, then I’m somewhat more hopeful. But it will take a better stance than rejection of ES4 “in whole [and] in part” to get somewhere.

Open letter to Chris Wilson


You seem to be repeating falsehoods in blogs since the Proposed ECMAScript 4th Edition Language Overview was published, claiming dissenters including Microsoft were ignored by me, or “shouted down” by the majority, in the ECMAScript standardization group. Assuming you didn’t know better, and someone was misinforming you, you (along with everyone reading this letter) know better now. So I’ll expect to see no more of these lies spread by you.

Everything that has been done to advance JavaScript to the proposed ECMAScript 4th Edition (ES4) in Ecma TC39-TG1 has been in the open for well over a year, first via the es4-discuss mailing list (complete archive) and exports of our group’s wiki, along with public talks I’ve given. Then by hosting the reference implementation for ES4 as proposed, along with our public trac database. And finally by the whole wiki being fully opened up and hosted live — including Microsoft’s “ES3.1” proposals.

These ES3.1 proposals came suddenly in March after years of passive participation by Microsoft, while the rest of the group worked on ES4. They have not been touched since April. They weren’t published via wiki export at that time, by agreement among all parties in TG1 including Microsoft and Doug Crockford. For one thing, they were (and still are) extremely rough and incomplete, with unanswered comments.

Almost all of the non-deprecation ES3.1 proposals were lifted directly from ES4 proposals (see my comment here).

One document reaffirms an agreement made in March: any ES3.1 candidate specification would be developed as a subset of the ES4 reference implementation, so it could be tested mechanically for forward and backward compatibility. This agreement has not been kept.

In February, an early warning was given by Microsoft, and met with vigorous debate (no shouting). Readers should take the time to follow this link, because I think it gets to the heart of the dispute, which is superficially over whether ECMAScript should remain small and mostly the same as it is in Edition 3. The pattern of general assertions about small being beautiful and sufficient for web application developers’ needs, met by specific arguments listing use-cases where JS does not scale in time or space, or lacks basic data integrity, type safety, and programming in the large support, in turn met by absolutely zero specific counter-arguments — this is a pattern we have seen over and over in TG1 this year.

The small-is-beautiful generalization alternates with don’t-break-the-web, again without specifics in reply to specific demonstrations of compatibility. Oh, and sometimes fix-security-first is put forth, without sound and sufficient solutions. At best, we have a fundamental conflict of visions and technical values between the majority and the minority. However, the obvious conflict of interest between the standards-based web and proprietary platforms advanced by Microsoft, and the rationales for keeping the web’s client-side programming language small while the proprietary platforms rapidly evolve support for large languages, does not help maintain the fiction that only clashing high-level philosophies are involved here.
Readers may not know that Ecma has no provision for “minor releases” of its standards, so any ES3.1 that was approved by TG1 would inevitably be given a whole edition number, presumably becoming the 4th Edition of ECMAScript. This is obviously contentious given all the years that the majority of TG1, sometimes even apparently including Microsoft representatives, has worked on ES4, and the developer expectations set by this long-standing effort.

A history of Microsoft’s post-ES3 involvement in the ECMAScript standard group, leading up to the overt split in TG1 in March, is summarized here.
The history of ECMAScript since its beginnings in November 1996 shows that when Microsoft was behind in the market (against Netscape in 1996-1997), it moved aggressively in the standards body to evolve standards starting with ES1 through ES3. Once Microsoft dominated the market, the last edition of the standard was left to rot — ES3 was finished in 1999 — and even easy-to-fix standards conformance bugs in IE JScript went unfixed for eight years (so three years to go from Edition 1 to 3, then over eight to approach Edition 4). Now that the proposed 4th edition looks like a competitive threat, the world suddenly hears in detail about all those bugs, spun as differences afflicting “JavaScript” that should inform a new standard.

Sorry, but most of those JScript deviations are not candidate de facto standards — they are just your bugs to fix. They should not stall ES4 for one second. I’m testifying as an eyewitness, and I have other witnesses in TG1 who can confirm what I write here. Everyone in the majority of TG1 gave time to the late-breaking dissenters from Microsoft and Yahoo!. We went out of our way to read, cross-reference, and comment on the “3.1” proposals. Activity on those proposals stopped in April.

Anyone who wants to check my claims against the TG1 meeting notes can go to the meetings page on the wiki, and follow the links. The face-to-face notes starting in March are easy to scan for political content (including my dissenting notes), and I invite anyone who is interested to have a look.

The meeting notes were edited by many hands, including Microsoft folks to their evident satisfaction, at the time of the meetings. The change logs are public in the wiki. There was no ignoring of minority positions. There was no failure to listen on the part of the majority. On the other hand I did not stop all progress on ES4 until unanimous agreement was reached on how to proceed (since Ecma and ISO do not require unanimity to make standards).

There certainly was no shouting down of the dissenters — that’s a bold lie in view of the well-attended and friendly dinners sponsored by the face-to-face meeting hosts. As the links above demonstrate, there has been no secrecy on the part of the majority of TG1, other than the secrecy requested by Microsoft and required by Ecma. Rather the opposite: Doug Crockford and Microsoft’s reps would depart for private meetings, in the middle of the March, April, and especially the May TG1 face-to-face meetings.

My TG1 Convenor’s Report, dated 4 May 2007, to Ecma TC39 (chaired by Jim Miller of Microsoft) is available to Ecma members at the Ecma members website. In it, I informed our parent TC39 committee that “[a] sub-group of TG1 is investigating possibly standardizing a smaller, compatible language, but this proposal has not yet been adopted.” Since then, the sub-group has failed to present a coherent proposal — JScript informative documents are not proposals — and my next report to Ecma will have to say so plainly.

You may have been misled, but now that everyone can see almost all of the primary sources (the remaining documents are hidden behind Microsoft, Yahoo!, and Ecma firewalls), please stop repeating falsehoods about either my conduct as TG1 Convenor, or the treatment of dissent by the majority favoring ES4 in TG1. If you have any documents to publish that shed light on Microsoft’s actions and intentions in Ecma TC39-TG1, I welcome them.


New Projects

My Ajax Experience West keynote covers a lot of ground, with slant-wise truth telling the over-arching theme. Mozilla believes in fairly radical open source action, including open strategy. In that spirit, three new projects, the first already known via jresig:

ActionMonkey, the project to join forces between SpiderMonkey and Tamarin, being driven by Jason Orendorff with Edward Lee and some old-timers and newcomers lending a hand. The first stage of development integrates MMgc into SpiderMonkey as a replacement for SpiderMonkey’s GC. (NB: the wannabe mascot has a day job.)

IronMonkey, the project to map IronPython and IronRuby to Tamarin, so that Tamarin becomes multi-lingual, over time delivering high performance for all languages. This effort is being spearheaded by Seo Sanghyeon (whose last name I apologize for misspelling — or for using a different spelling of, at any rate — in my slides). The idea is to make use of the MsPL-licensed open source that Microsoft has kindly given the world, and tend necessary changes or additional code in downstream truly-open repositories that can accept patches from non-MS employees, such as FePy.

ScreamingMonkey, the project to add browser-specific script-engine glue to Tamarin, enabling it as a <script> tag handler in other browsers, starting with IE. Mark Hammond is leading this charge. Where it will end, one can only speculate. At a minimum, if you have Tamarin installed on your system in the future, you may need only a small download to plug it into other browsers. Best case, it comes with all the relevant glue in its main distribution. We’ll see.

I’m excited to announce these projects. They address various concerns voiced by bloggers of the form “what about other languages?” and “what about IE?”. Comments welcome.


ECMAScript Edition 4 Reference Implementation

As Dave Herman just posted at Lambda-the-Ultimate, we ECMA members who have been working on the successor to the JavaScript standard have a new website, which now hosts the “milestone 0” release of the ES4 Reference Implementation, written in Standard ML of New Jersey.

As Dave notes, this is a “pre-release” in the Open Source, “release early and often” sense. We are not done yet, but the Reference Implementation passes over 90% of the ES3 tests that we host in I should note here that the Reference Implementation already handles a great deal of ES4 in addition to most of ES3, so for example the self-hosted built-in classes are mostly there.

We know of some bugs, and we welcome well-written reports of more to fix. If you observe something that might be a bug, but you aren’t ready to file a Trac ticket, feel free to ask for advice on the es4-discuss list.

Extravagant praise to Dave for setting up the site and working on all the details of hosting the source files and binaries. Also to Graydon for tremendous work creating and developing much of the SML code. SML has been a good choice for us, although the performance of our (intentionally not very optimized) reference code can be poky. However, Dave says he is working on a MLton port, which should give a nice performance boost (MLton is an optimizing whole-program SML compiler).

This is just the beginning. Our plan is to finish the Reference Implementation over the summer and then work on specification language to surround pretty-printed excerpts of the SML and self-hosted ES4 code. At the same time, Mozilla, Adobe, and anyone who wants to help will bring up the new language on Tamarin in the new Mercurial repository. I’ll have more to say about that in a bit.


Open Source and “Openness”

Adobe open-sources Flex under the MPL — a good second step. Congratulations to all the people who made it happen.

My browser-war battle-heightened senses tell me you’ll see more open-sourcing, and no small amount of “openness” hype, from several big players in the coming days. As we mozilla.organs learned the hard way, getting the license right and hosting the code in a public repository are not enough (no pixie dust).

Successful open source projects combine meritocratic leadership, “doing” more than “talking”, and breadth through well-scoped extension mechanisms. It’s not enough to do great work by oneself: each committer who has the stamina and remains engaged must spend time listening to users and developers, grooming helpers and successors, and refactoring or even redesigning to support what becomes, module by module, a platform.

Platforms are for applications to stand upon, for building higher tiers of platform, and ultimately (in my view) for enabling user innovation. Mozilla’s platform has at least three tiers: Web, XUL, Gecko-embedded. Firefox has addons as a tier distinct from those.

Of course, closed-source platforms exist, but the successful ones hold up so much code that they end up de-facto standards, and often go open-source just to reduce costs born by the proprietor (there are better reasons than that, don’t get me wrong).

One way of evaluating Adobe’s success in open-sourcing Flex will be to look for more than just added components and widgets, and discern higher-level as well as different platform tiers. For instance, would patches to add a web-targeted compiler back end, in addition to the Flash Player SWF-targeted back end that’s there now, be accepted if they were developed?

Or how about harmonizing XUL and MXML, making changes to underlying layout and widget models on both sides in order to converge the two, so that (with trivial XML vocabulary translation) MXML could be interpreted by browsers that support XUL, as well as compiled into a SWF bound for the Flash Player?

For many observers and fans, an open-source Flash Player is the holy grail. If that is coming, it will probably take a while. But there’s a small and (I think) rising chance that “openness” competition among big players will speed up the trend at Adobe.

Then what? Obviously, forked and fragmented Player variants are not helpful, and the main distribution will be the contested ground. A better Flash Player via open-source development would be fine: good not great.

Blending Flash code into multiple browser rendering pipelines, with seamless text input (including focus) and output, an integrated DOM, full and fast scripting, and unified CSS processing would be even better. Add high-quality, unencumbered <video>, and web-friendly 3D (including an LOD-capable scene-graph model, rather than a bulky, eagerly created DOM), and you’re close to my idea of the promised land.

Adobe is moving in the right direction. That’s good enough for me today.

The Open Web and Its Adversaries

A small presentation I gave at SXSW asks what it means to be “Open” in the sense of open standards, open source, and a web whose major content formats are not controlled by a single vendor (I credited Nat Friedman’s amusing keynote from last year’s Free Software & Open Source Symposium for inspiration of the OED historical “Open” example).

Ted Leung (see followups) blogged about this topic already, specifically in connection with Adobe.

Ted refers to Anne Zelenka’s Why Open is Good and How Open Could Be Good for Flash post, which is worth reading, including the comments. In one comment, John Dowdell of Adobe asks whether the essence of “Open” is the ability to fork. Anne notes the loaded connotation of “fork” but agrees that John’s analysis is “not incorrect.”

I would go further. Forking is an extreme point in a continuum of options that exist with open source. The option to fork must exist as a feedback mechanism, but it need not be used in order for users to gain benefits not available with closed source and proprietary standards. Forking can be the right thing, or it can be a kind of mutually-assured-destruction option that keeps everyone acting in the interest of not forking.

Prior to even approaching a fork, open standards and open source both empower user-driven innovation. This is old news to the Mozilla user community, who have been building and feeding back innovations for the life of the project, increasing over time to include Firefox add-ons and GreaseMonkey user scripts. (BTW, I am pushing to make add-on installation not require a restart in Firefox 3, and I intend to help improve and promote GreaseMonkey security in the Firefox 3 timeframe too.) Without forking, even to make private-label Firefoxes or FlashPlayers, users can innovate ahead of the vendor’s ability to understand, codify, and ship the needed innovations.

Consider just the open standards that make up the major web content languages: HTML, CSS, DOM, JS. These mix in powerful ways that do not have correspondences in something like a Flash SWF. There is no DOM built inside the FlashPlayer for a SWF; there’s just a
display list. There’s no eval in ActionScript, and ActionScript features a strict mode that implements a static type checker (with a few big loopholes for explicit dynamic typing). You can’t override default methods or mutate state as freely as you can in the browser content model. Making a SWF is more like making an ASIC — it’s “hardware”, as Steve Yegge argues.

This is not necessarily a bad thing; it’s certainly different from the Open Web.

I assert that there is something wrong with web-like “rich” formats that aren’t hyperlink-able or indexable by search-engines. You could argue that these bugs could be fixed, and Flash is wisely becoming more URI-addressable and view-source-able over time. But it still ain’t the Web. It is not hand-authored, easily tweaked incrementally, copy-and-paste-able. It’s

Given the stagnation of the web under IE and the failure of Java on the client, there’s a place for Flash, and I’m sure Microsoft would argue that means there’s a place for WPF/E (but I don’t see the point: either bet on the Web, or bet against it in a transcendently different way from trying to overtake Flash). If I were VP of Engineering in a 10,000 person company, I would want the security blanket of the C-like syntax and a static type system for a well-known, big-bell-curve language like AS3, C#, or Java.

And indeed AS3, C#, and Java all are marketed to big companies with hordes of “hardware” hackers creating fixed-point UIs. Dead UIs, in other words, not enlivened by user innovation. And sorry to say (to Yahoo! and Google, who may be exceptions to the rule), the big innovations don’t come from the big dumb companies.

Meanwhile, the Web is alive precisely because of the distributed extensibility of its content languages, both on the server side and in the client JS/DOM world, not to mention add-ons and GreaseMonkey.

Dare Obasanjo argues that developers crave single-vendor control because it yields interoperation and compatibility, even forced single-version support. Yet this is obviously not the case for anyone who has wasted time getting a moderately complex .ppt or .doc file working on both Mac and Windows. It’s true for some Adobe and Microsoft products, but not all, so something else is going on. And HTML, CSS, DOM and JS interoperation is better over time, not worse. TCP/IP, NFS, and SMB interoperation is great by now. The assertion fails, and the question becomes: why are some single-vendor solutions more attractive to some developers? The answers are particular, not general and implied simply by the single-vendor condition.

Implicit in my writing is the assumption (conclusion, really) that browsers can adopt the necessary advanced rendering and faster virtual-machine programming language support that the “rich client platforms” boast (or promise in version 2.0). For instance, we will support an OpenGL-ES binding for canvas in the near term, in Firefox 3 if vlad and I can help it. There’s no technical reason this can’t be interoperably supported by other browsers in the near term.

Can the Open Web keep improving enough to stave off the Closed Web? Of course, Mozilla is here to help, and we want every browser to support the nascent standards we support. But at some point the answer does depend on Microsoft upgrading IE without forking web standards from what Firefox, Safari, and Opera agree on now and in their upcoming releases, which are not yet implemented in IE7.

This will mean incompatibility for old IE content, or more painful version checking in the IE engine, or a mix. Note how Flash 9 includes both the AS2 and AS3 (Tamarin) engines, since Adobe could not keep compatibility in one engine. Gecko has had to put up with more de-facto quirks implementation than we might prefer; we sucked it up to get market share. IE can do likewise, for a change to avoid losing more market share.

Such a brute-force versioning approach could work well for Microsoft, as it did for Adobe in Flash 9. Microsoft can most afford to put more megabytes on disk via Windows Update than can browser vendors such as Mozilla and Opera, who rely on
voluntary downloads.

Anyway, I’m committed to improving the Open Web, even at the expense of Firefox market share if it comes to it (but we are still growing, and Firefox still has the second-place leverage to improve the Web quickly). If we should fail and just make a fancier browser whose nascent standards are not adopted by IE, at least we tried. The alternative is to renounce innovation and let the proprietary rich clients move the Closed Web forward. That might be convenient for some (big company) developers. It’s not in the interest of the larger developer community, or of the public, in my book.

Let me know what you think.

Threads suck

This is not an original thought, but I write with some authority here.

I hacked Unix kernel code out of grad school at SGI, in SGI’s “good old days” (1985-1992). Among other things, we took single-threaded (ST) kernel code and multi-threaded (MT) it on SGI’s SMP boxes. I won a free trip to New Zealand and Australia in 1990 along with Bent Hagemark, with kernel source on magtape in our hot little hands, on account of others’ bugs in adapting some single-threaded (ignoring interrupts) AT&T “Streams” code to SGI’s SMP kernel, and made fixes in the field (thanks to the SGI sales guys in Brisbane, we even got two nights on the Gold Coast in compensation — not long enough!).

You must be this tall to hack on threaded systems, and that means most programmers should run away crying. But they don’t. Instead, as with most other sharp tools, the temptation is to show how big one is by picking up the nearest ST code and jamming it into a MT embedding, or tempting race-condition fate otherwise. Occasionally the results are infamous, but too often, with only virtual fingers and limbs lost, no one learns.

Threads violate abstractions six ways to Sunday. Mainly by creating race conditions, deadlock hazards, and pessimistic locking overhead. And still they don’t scale up to handle the megacore teraflop future.

We can hope for better static analyses to find all races. In the real world, the code is C or C++ and there’s no hope for static salvation. Sure, some languages try to put deadlocks in a syntactic cage, but that walks right into the overhead problem, in spite of heroic VM-based optimizations. Unexpected costs, even if constant or linear, can sink any abstraction. For example (still piquant to Mozilla hackers busy deCOMtaminating), virtual method calls cost; they should be avoided where you’re writing hardware. The same goes for locks: not all abstractions must be MT-safe; some must be ST and fast.

So my default answer to questions such as the one I got at last May’s Ajax Experience, “When will you add threads to JavaScript?” is: “over your dead body!”

There are better ways. Clueful hackers keep rediscovering Erlang. Then there is STM. One retro stylist I know points to an old language-based solution, Hermes.

A requirement for JS3 (along with hygienic macros) is to do something along these more implicit lines of concurrency support. In all the fast yet maintainable MT systems I’ve built or worked on, the key idea (which Will Clinger stated clearly to me over lunch last fall) is to separate the mutable unshared data from the immutable shared data. Do that well, with language and VM support, and threads become what they should be: not an abstraction violator from hell, but a scaling device that can be composed with existing abstractions.

So here’s a promise about threads, one that I will keep or else buy someone a trip to New Zealand and Australia (or should I say, a trip here for those over there): JS3 will be ready for the multicore desktop workload.

Does this mean we won’t be hand-coding MT Gecko? I think Graydon said it best: “I’d rather eat glass.” Most high-level concurrent Mozilla programming will be in JS3 on the back of an evolved Tamarin.

Oink-based Patch Generation

For Mozilla 2, I proposed that we use Oink, or really Elsa, to build a tool that can help automate deCOMtamination, switching to C++ exceptions instead of nsresults, and similar tasks beyond the reach of sed or perl. The idea is to specify pattern and replacement abstract syntax trees (ASTs), and have the tool match the pattern against sub-tree instances, producing a version of the source with the replacement AST pretty-printed instead of the matching sub-tree.

Graydon came up with the insight of using source coordinates, which the Oink framework tracks accurately, to tell which lines in a given primary source (.cpp file, not .i file that results from the C pre-processor) correspond to a particular AST. The tool would read the source, counting lines until the range to delete is reached, then copy each such line matching the pattern to replace with a “- ” prepended. Then write lines pretty-printed from the replacement AST with “+ ” prepended. Add a bit more boilerplate and context lines, and you have a patch.

Graydon didn’t just make this breakthrough, he prototyped it, with a hardwired pattern to replace struct foo pointers with struct bar pointers in a test file:

$ cat
struct foo {};
struct holdsAFoo
char x;
static const foo * const y;
int *z;
$ ./staticprint -fs-print-rewrite
@@ -5,1 +5,1 @@
-  static const foo * const y;
+static struct bar const * const y;
$ ./staticprint -fs-print-rewrite | patch -p0
patching file
$ cat
struct foo {};
struct holdsAFoo
char x;
static struct bar const * const y;
int *z;

All this wants is an Oink tool of its own (instead of a hack to staticprint), a little more indentation fidelity (bonus points for respecting the source file’s Emacs and vim modelines), C++ purity (no struct before bar needed), patch context and hunk coalescing, and generalized AST pattern/replacement options, and we’ll be able to make major systematic changes by generating patches. Neat!


Project Tamarin

I just committed the initial revision of mozilla/js/tamarin, the open-source ActionScript Virtual Machine contributed by Adobe. But I had nothing to do with this fine work. It’s the product of some talented folks at Adobe (originally at Macromedia; at least one of whom has moved on and is doing a startup).

Tamarin, the open-source AVM, is the JIT-oriented JavaScript VM that I mentioned in my last post. We have already developed, via a two-day marathon hacking session, a set of proof-of-concept patches to integrate it with SpiderMonkey. The resulting Franken-VM was compiling and linking when I last laid hands on it, but not quite limping. This exercise produced many insights to guide more durable integration. And it was great fun interacting with the new open-source contributors from Adobe, whom I would like to welcome to Mozilla:

  • Jeff Dyer, self-hosting compiler architect
  • Steven Johnson, Tamarin developer
  • Tom Reilly, garbage collector developer
  • Rick Reitmaier, JIT developer
  • Dan Smith, Tamarin module owner
  • Edwin Smith, Tamarin creator and VM architect
  • Erik Tierney, Tamarin developer

My apologies to those whose names I haven’t yet learned. I’ve had the pleasure of getting to know most of the above-named folks through the ECMA TG1 (JavaScript standards) group. They are good people and skillful hackers, and I’m grateful that they are adding their strength to the Mozilla project.

I should point out that, although Adobe did not release open source for its existing, free AS3 compiler, it did contribute a self-hosting (AS3-in-AS3) compiler prototype written by Jeff Dyer. We intend to develop this jointly into an ES4/JS2 self-hosting compiler, implementing the full and final ECMA-262 Edition 4 spec when it is done. In the mean time, if you want to generate ABC for Tamarin, you have open source options.

I’ll be on IRC in a little over nine hours to chat about this great news and answer questions, and then I’m off to Web 2.0. Later this week I will blog again about some early progress on another Mozilla 2 front.


Update: Frank Hecker of the Mozilla Foundation clearly and concisely answers many basic questions about What This All Means, questions to which I tend to give short shrift in this too-technical blog of mine. Recommended.