Herewith a hacked-up version of my S5 slides, with notes and commentary interpolated at the bottom of each slide.
See how JS is paired with Flash — poor, mundane HTML, CSS, DOM! HTML5 needs a new name.
I described Doug Crockford as “the Yoda of Lambda JavaScript programming” at a mid-2006 talk he invited me to present at Yahoo!, so I thought I would start by riffing on whether the role still fits. So far, so good.
But three prequels later, the outlook for Yoda is not good. Large, heavy, spinning, flying-saucer-proprietary runtimes are hurtling toward him!
Enough Star Wars — Doug’s too tall for that part (even if it’s only a muppet). Let’s try a taller wizard…
I don’t really believe ES4 is a demon from the ancient world, of course. I’m afraid the JS-Hobbits are in trouble, though. As things stand today, Silverlight with C# or something akin (pushed via Windows Update) will raze the Shire-web, in spite of Gandalf-crock’s teachings.
I can roleplay too: let’s see, renegade veteran from the losing side of an epic war against an evil empire… yeah, I can relate.
I really do think that JS’s multi-paradigm nature means there is no one-true-subset for all to use (whether they like it or not), and the rest — including evolutionary changes — should be kept out. I reject the idea that instead of making JS better, programmers should somehow be made “better”. The stagnation of JS1, and “little language” idolatries surrounding it (those one-true-way JS subsets), impose a big tax on developers, and drive too many of them away from the web standards and toward WPF, Flex, and the like.
Ok, enough role-playing geek fun — let’s get down to brass tacks: what’s really going on with the ES4 fracas?
Clearly ES4 will be tough to standardize. Standards often are made by insiders, established players, vendors with something to sell and so something to lose. Web standards bodies organized as pay-to-play consortia thus leave out developers and users, although vendors of course claim to represent everyone fully and fairly.
I’ve worked within such bodies and continue to try to make progress in them, but I’ve come to the conclusion that open standards need radically open standardization processes. They don’t need too many cooks, of course; they need some great chefs who work well together as a small group. Beyond this, open standards need transparency. Transparency helps developers and other categories of “users” see what is going on, give corrective feedback early and often, and if necessary try errant vendors in the court of public opinion.
Given all the challenges, the first order for ES4 work is to finish the reference implementation and spec writing process, taking into account the ongoing feedback. Beyond that, and I said this at the conference, I believe we need several productized implementations well under way, if not all but done, by the time the standard is submitted for approval (late 2008). This will take some hard work in the next ten months.
My hope is to empower developers and users, even if doing so requires sacrifice on the part of the vendors involved.
Some assert that JS1 is fine, browsers just need better APIs. Or (for security), that JS1 with incompatible runtime semantic changes and a few outright feature deletions is fine, but mainly: browsers just need better APIs. Or that Security comes first, and the world should stop until it has been achieved (i.e., utopia is an option). But I contend that JS must improve along with browser APIs and security mechanism and policy, both to serve existing and new uses, and to have a prayer of more robust APIs or significantly better security.
It’s clear from the experiences of Mozilla and security researchers I know that even a posteriori mashups built on a capability system will leak information. So information flow type systems could be explored, but again the research on hybrid techniques that do not require a priori maximum-authority judgments, which do not work on the web (think mashups in the browser without the user having to click “OK” to get rid of a dialog), is not there yet. Mashups are unplanned, emergent. Users click “OK” when they shouldn’t. These are hard, multi-disciplinary research problems.
Where programming languages can help, type systems and mutability controls are necessary, so JS1 or a pure (semantics as well as syntax) subset is not enough.
I am personally committed to working with the Google Caja team, and whoever else will help, to ensure that JS2 (with the right options, and as few as possible) is a good target for Caja. The irony is that when combined with backward compatibility imperatives, this means adding features, not removing them (for example, catchalls).
A note on names: I used JS2 in the title and the slides to conjure with the “JS” and “JavaScript” names, not to show any disrespect to ES/ECMAScript. All the books, the ‘J’ in AJAX (when it’s an acronym), the name of the language dropped most often at the @media Ajax conference, all call the language by the “JavaScript” name. Yeah, it was a marketing scam by Netscape and Sun, and it has a mixed history as a brand, but I think we are stuck with it.
This slide compresses a lot, but makes some points often missed by fans of other languages. Browsers will always need JS. Browsers cannot all embed the C Python implementation, the C Ruby implementation, etc. etc. — code footprint and cyclic leaks among heaps, or further code bloat trying to super-GC those cycles, plus all the security work entailed by the standard libraries, are deal killers.
The multi-language, one-runtime approach is better, but not perfect: IronPython is not Python, and invariably there is a first-among-equals language (Java on JVMs, C# on the CLR). We are investing in IronMonkey to support IronPython and IronRuby, and in the long run, if everyone makes the right moves, I’m hopeful that this work will pay off in widespread Python and Ruby support alongside JS2. But it will take a long while to come true in a cross-browser way.
Silverlight is not able to provide browser scripting languages in all browsers. Even if IE8 embeds the DLR and CLR, other browsers will not. Note the asymmetry with ScreamingMonkey: it is likely to be needed only by IE, and only IE has a well-known API for adding scripting engines.
The “too small” and “too few primitives” points remind me of Guy Steele’s famous Growing a Language talk from OOPSLA 1998 (paper). If you haven’t seen this, take the time.
During the panel later the same day, Jeremy Keith confronted me with the conclusion that JS2 was pitched only or mainly at Java heads. I think this slide and the next gave that impression, and a more subtle point was lost.
I hold no brief for Java. JS does not need to look like Java. Classes in JS2 are an integrity device, already latent in the built-in objects of JS1, the DOM, and other browser objects. But I do not believe that most Java U. programmers will ever grok functional JS, and I cite GWT uptake as supporting evidence. This does not mean JS2 panders to Java. It does mean JS2 uses conventional syntax for those magic, built-in “classes” mentioned in the ES1-3 and DOM specs.
In other words, and whatever you call them, something like classes are necessary for integrity properties vital to security in JS2, required for bootstrapping the standard built-in objects, and appropriate to a large cohort of programmers. These independent facts combine to support classes as proposed in JS2.
JS1 is used by non-programmers, beginning programmers, “front end designers”. It is copied and pasted, or otherwise concatenatively programmed, with abandon (proof: ES4 has specified toleration of Unicode BOMs in the middle of .js files! How did those get there?). This was a goal, at least in the “pmarca and brendan”@netscape.com vision circa 1995, over against Java applets. We succeeded beyond our wildest nightmares.
Netscape 2 had some very clueful early adopters of JS (Bill Dortch, if you are reading this, leave a comment). Years later, Doug Crockford led more select hackers toward the right end of the distribution, but much of the middle was necessarily bypassed: you can’t reach this cohort without taking over Java U.
What’s more, I observe that the Lambda-JS Jedi order is inherently elitist (I’m not anti-elitist, mind you; natural elites happen in all meritocratic systems). For many current Knights, it must remain so to retain its appeal.
Now, there’s nothing wrong with using closures for (partial) integrity and prototypes for inheritance; I like these tools (I should, I picked them in a hurry in the early days). But really, why should everyone be required to learn the verbose, error-prone, and inherently costly functional-JS-OOP incantations (power constructors, module patterns, etc.), instead of using a few concise, easier to get right, and more efficient new declarative forms that JS2 proposes?
It’s not as if JS2 is renouncing prototypes or closures in favor of “the Java way”. That’s a misinformed or careless misreading. Rather, we aim to level the playing field up, not down. JS2 users should be able to make hardened abstractions without having to write C++ or create Active X objects. And power-constructor and module pattern fans can continue to use their favorite idioms.
JS1 favors closures (behavior with attached state) over objects (state with attached behavior) with both more abstraction (because names can be hidden in closures) and greater integrity (because var bindings are DontDelete
). While JS1 is multi-paradigm, going with the grain of the design (closures over objects) wins. In my talk, I acknowledged the good done by Doug and others in teaching people about functional programming in JS.
However, there are limits. JS1 closure efficiency, and outright entrainment hazards that can result in leaks, leave something to be desired (the entrainment hazards led Microsoft to advise JScript hackers to avoid closures!). You could argue that implementations should optimize harder. Arguing is not needed, though — high quality optimizing runtime, which fit on phones, are what’s needed.
Beyond efficiency, using closures for modules and class-like abstractions is verbose and clumsy compared to using new syntax. Dedicating new syntax is scary to some, but required for usability (over against __UGLY__
names), and allowed under script type versioning. In the absence of macros (syntactic abstraction), and so long as macros can be added later and syntactic sugar reimplemented via macros at that time, my view is that we should be generous with syntactic conveniences in JS2.
So JS2 has more syntax (user interface) as well as substance (complementary primitives). This could be a burden on people learning the new language, but I think not a big one. In practice over the next few years, the bulk of the focus in books and classes will be on JS1 and Ajax. A programming language for the web should be a many-storied mountain, and most users will not ascend to the summit.
The main burden of new syntax is on folks writing source-to-source translators, static analyzers, and the like. These more expert few can take the heat so that the many can enjoy better “user interface”.
To respond to the contrarian arguments in the previous slide:
- Whoever prefers a subset is free to continue using it on JS2 implementations. If your pointy-haired boss imposes class-obsessed B&D programming on you, get a new job.
- DOM profiles show high dynamic dispatch, argument/result conversion, and other costs imposed by untyped JS1 in current implementations. Better implementations and JS2 usage can help.
- Security is best defined as end-to-end properties that must be engineered continuously according to economic trade-offs as the system evolves. Utopia is not an option.
- Some people are learning, but many others are not, and vendors sell C# and AS3 against JS1 for good reason.
As Neil Mix wryly observed in a post to the es4-discuss list:
When I hear concerns that ES4 will “break the web,” I can’t help but think of how many times I’ve heard that the web is already broken! The risks of not adopting ES4 surely must factor into this calculus, too.
At the very least, don’t put all eggs in the “make people better” basket.
I get all kinds of fan-mail :-/.
The argument that stagnation on the web fostered Ajax and Web 2.0 is false. Only when XMLHttpRequest
was cloned into other browsers, and especially as Firefox launched and took market share back from IE, did we see the sea-change toward web apps that rely on intensive browser JS and asynchronous communication with the server.
Like Mal Reynolds, I really do believe these things — not that I’m ready to die for my beliefs (“‘Course, that ain’t exactly plan A”).
There won’t be a single browser release after which all power shifts to web developers writing in JS2. The process is more gradual, and it’s easy to forget how far we’ve come already. We’re well along the way toward disrupting the desktop OSes. Yet JS2 on optimizing VMs will liberate developers in ways that JS1 and plugins cannot.
I lived through the Java dream. Netscape was building an optimizing JIT-compiling runtime in 1996 and 1997, while Sun acquired Anamorphic and built HotSpot. The layoffs at the end of 1997 brought all the Netscape Java work crashing to a halt, and caused the “Javagator” rendering engine team to reinvent their code in C++ as Gecko (originally, “Raptor”).
In spite of all the hype and folly, the dream could have come true given both enough time and better alignment (including open source development) between Sun and Netscape. A lot more time — Java was slow back then, besides being poorly integrated into browsers.
There are many ironies in all of this. Two I enjoy often:
- Flash filled the vacuum left by the decline of Java in the browser, and now provides a vector for JS2 on Tamarin.
- Microsoft dumped Java, depriving Outlook Web Access of an asynchronous I/O class, wherefore
XMLHttpRequest
.
If you take only one point away from this talk (I said), it should be that type annotations are not required for much-improved pure JS performance.
This chart shows results, normalized using SpiderMonkey performance at unity (so taller is faster), for the JavaGrande benchmark ported to JS (untyped JS except where noted: “Tamarin with Type Annotations”). The “Trace-Tree JIT” blue bars show results for a clever translation of Tamarin bytecode into Java bytecode (with runtime type helpers) fed into a tracing JIT implemented in Java(!). Amazingly, this approach competes with Rhino and Tamarin, even Tamarin run on typed-JS versions of the benchmarks.
The Crypt benchmark could not be run using the trace-based JIT at the time this chart was generated.
More good results, especially given the preliminary nature of the research. With an x86 back-end instead of the Java back-end used for these benchmarks, and further tuning work, performance should go up significantly. Even at this early stage, Series, SOR, and SparseMatMult all show the tracing JIT working with untyped JS beating Tamarin on typed-JS versions of these benchmarks.
In response to an informal recap of my presentation the other day, Rob Sayre mentioned Peter Norvig‘s presentation on design patterns in dynamic programming. This caused me to flash back to the bad old days of 1998, when certain “Raptor” architects would wave the Gamma book and make dubious assertions about one-true-way design patterns in C++.
Norvig’s slides show what was lost by the swerve toward static, obsessively classical OOP in C++ and Java, away from dynamic languages with first-class functions, first-class types, generic methods, and other facilities that make “patterns” invisible or unnecessary. JS2 aims to restore to practical programmers much of what was lost then.
The Romans called wheat integrale, referring to the potent and incorruptible completness of the kernel. Integrity as a security property is not far removed from this sense of completeness and soundness. JS1 simply lacks crucial tools for integrity, and JS2 proposes to add them.
The following slides (I’ve coalesced multiple slides where possible) show the evolution of a webmail library from JS1 to JS2, via gradual typing, in order to increase integrity and simplify code, avoiding repetitious, error-prone hand-coded latent type checking. The transport code is omitted, but you can see JSON APIs being used for transfer encoding and decoding.
It’s rare to see anything in real-world JS like the detailed checking done by validateMessage
. It’s just too tedious, and the language “fails soft” enough (usually), that programmers tend to skip such chores — sometimes to their great regret.
Important points:
- Structural types are like JSON, but with types instead of values
- The
like
type prefix makes a “shape test” spot-check - Note how
fetch
is now an expression closure - No more
validateMessage
! Structural types ftw! 🙂
This stage copes with a confused or malicious client of the webmail API, who could mutate a reference to a message to violate the validity constraints encoded in stage 1’s validateMessage
.
The MsgNoId
type allows the library client to omit a dummy id
, since send
initializes that property for the client.
Note the righteous use of parameter and let
destructuring in copyMessage
.
wrap
is both an annotated type prefix and a binary operator in JS2. It makes a wrapper for an untyped object that enforces a structural type constraint on every read and write, in a deep sense. So instead of copying to provide integrity through isolation, this alternative third stage shares the underlying message object with the library client, but checks all accesses made from within the webmail library.
Notice the lack of classes so far.
UPDATE: A revised and extended version of this evolutionary arc is now available as a tutorial at ecmascript.org, with compelling side-by-side comparisons of successive stages.
The “typed APIs with untyped code” pattern is particularly winning in our experience building the self-hosted built-ins in the ES4 reference implementation.
During this slide, I shot screaming slingshot flying monkeys (complete with little black masks and capes) into the audience. I’m sorry I could bring only a handful on this trip!
Chris provides source code, Firefox 3 alpha builds, and Ogg video/audio files.
Now available as a Firefox 3 addon!
I forgot to note: best viewed in Firefox 3 beta 1 or nightly, for correct display:inline-table rendering.
/be
“Browsers cannot all embed the C Python implementation”
No one’s really asking for CPython exactly; you obviously can’t make most of the standard library available.
So when you say “IronPython is not Python”, that’s true, but CPython in the browser is not Python (in that sense) either.
I think what people want is the nice clean syntax, straightforward class structure, and powerful tools like generators that the Python 2.5 language has, not necessarily the implementation.
IronPython is a fine way to do that.
Colin: “nice clean syntax, straightforward class structure, and powerful tools like generators” sounds like JS2, to some ears. My words were in reply to people who advocate strongly for the direct embedding of the canonical “implementation-as-spec” version of a popular language (e.g. Lua).
If IronPython is really the right answer, as I allowed in my words about IronMonkey, then we have that case covered.
/be
I feel like I just read a John Carmac “.plan” file. Fascinating, but way over my head. 🙂
After church this past Sunday during cleanup I happened to hear (not see — that was second!) one of the very flying monkey toys you mention here being deployed by a random kid, Brendan, and you should be pleased to know that my *immediate* first thought was “ScreamingMonkey”! I had to smile a bit at the unexpected cognitive clash of worlds.
I’m really excited about JS2!
People have been bad-mouthing Java for quite some time now, but so far, there really haven’t been any alternatives when it comes to productive cross-platform development. I understand the excitement about Python and Ruby, but without some kind of static typing, I doubt there will ever be a Python or Ruby IDE that is as good as the Java IDEs.
On the other hand, there are languages such as Common Lisp, Dylan and Smalltalk and it makes me weep when I think about how old and how brilliantly designed they are.
To me, JS2 is the best (probably more like “good enough”; its syntax is a necessary compromise) of both worlds: Its optional type system will lead to nice IDEs, while features such as multiple dispatch, closures and generators greatly increase the expressiveness compared to Java.
There are two things that are still on my wish list for JS2:
(1) Mixins. Having class fragments one can mix and match to create classes is very useful.
(2) The standard library has to be fairly comprehensive. Maybe this is not as urgent in a browser setting, but JS2 should really become a general-purpose programming language. In Java, it pains me how often I have to write something myself (or download it) when the same functionality is readily available in the Python standard library.
So to the JS2 guys: Hang on in there, it’s a worthy cause. JS2 has huge potential in browser, server and desktop environments.