Search Results

8 Jun 2007

ECMAScript Edition 4 Reference Implementation

As Dave Herman just posted at Lambda-the-Ultimate, we ECMA members who have been working on the successor to the JavaScript standard have a new www.ecmascript-lang.org website, which now hosts the “milestone 0″ release of the ES4 Reference Implementation, written in Standard ML of New Jersey.

As Dave notes, this is a “pre-release” in the Open Source, “release early and often” sense. We are not done yet, but the Reference Implementation passes over 90% of the ES3 tests that we host in cvs.mozilla.org. I should note here that the Reference Implementation already handles a great deal of ES4 in addition to most of ES3, so for example the self-hosted built-in classes are mostly there.

We know of some bugs, and we welcome well-written reports of more to fix. If you observe something that might be a bug, but you aren’t ready to file a Trac ticket, feel free to ask for advice on the es4-discuss list.

Extravagant praise to Dave for setting up the site and working on all the details of hosting the source files and binaries. Also to Graydon for tremendous work creating and developing much of the SML code. SML has been a good choice for us, although the performance of our (intentionally not very optimized) reference code can be poky. However, Dave says he is working on a MLton port, which should give a nice performance boost (MLton is an optimizing whole-program SML compiler).

This is just the beginning. Our plan is to finish the Reference Implementation over the summer and then work on specification language to surround pretty-printed excerpts of the SML and self-hosted ES4 code. At the same time, Mozilla, Adobe, and anyone who wants to help will bring up the new language on Tamarin in the new Mercurial repository. I’ll have more to say about that in a bit.

/be

22 May 2006

JavaScript 2 / ECMAScript Edition 4

The talk I gave last Friday at XTech 2006 in Amsterdam, a revised and extended version of the talk I gave at the Ajax Experience the previous week in San Francisco, is here.

8 Oct 2012

Harmony of Dreams Come True

This blog focuses on portions of the new-in-ES6 stuff I presented in my Strange Loop 2012 closing keynote, which was well-received (reveal.js-based HTML slides, some from my Fluent 2012 keynote, many of those originally from Dave Herman‘s Web Rebels 2012 talk [thanks!], can be viewed here; notes courtesy Jason Rudolph).

UPDATE: the Strange Loop keynote video is up.


I blogged early in 2011 about Harmony of My Dreams, to try to fit in one page some dream-sketches (if not sketchy dreams — the #-syntax ideas were sketchy) of what I thought were the crucial elements of ECMAScript Harmony, the name I coined for the standardized future of JavaScript.

Now this dream is coming true, not just in ES6 draft specs but in prototype implementations in top browsers. Here I’ll tout Firefox 15, which released almost six weeks ago (yes, this means Firefox 16 is tomorrow, and Firefox 17 beta and 18 aurora too — these all have yet more new goodies in them — isn’t Rapid Release fun?). Per the MDN docs, the SpiderMonkey JS engine shipped in Firefox 15 sports the following new prototype-implemented draft ES6 features:

Default parameters

This extension (AKA “parameter default values”) is too sweet, and it will help put the arguments object out to pasture:


js> function f(a = 0, b = a*a, c = b*a) { return [a, b, c]; }
js> f()
[0, 0, 0]
js> f(2)
[2, 4, 8]
js> f(2, 3)
[2, 3, 6]
js> f(2, 3, 4)
[2, 3, 4]

Implementation credit goes to Benjamin Peterson for his work implementing default parameters, and to Jason Orendorff for his always-excellent code reviews. See this bug for followup work to track the latest ES6 agreement on how passing undefined (and only undefined) should trigger defaulting.

We have a few details to iron out still about scope, I suspect (based on this es-discuss message and its thread).

Rest parameters

Even sweeter than default parameters are rest parameters, and I bet they are ahead of default parameters in making arguments a bad memory some fine day:


js> function f(a, b, ...r) { print(Array.isArray(r)); return r.concat(a, b); }
js> f(1, 2)
true
[1, 2]
js> f(1, 2, 3)
true
[3, 1, 2]
js> f(1, 2, 3, 4, 5)
true
[3, 4, 5, 1, 2]

Again credit goes to Benjamin and Jason for their work.

Spread in array literals

The dual of rest is called “spread”, and it should work in call expressions as well as array literals. The latter is implemented in Firefox 16 (now in the beta channel):


js> a = [3, 4, 5]
[3, 4, 5]
js> b = [1, 2, ...a]
[1, 2, 3, 4, 5]

Thanks once again to Benjamin (a star Mozilla intern this summer) and Jason.

Spread in call expressions is not yet implemented:


js> function f(...r) { return r; }
js> function g(a) { return f(...a); }
typein:20:0 SyntaxError: syntax error:
typein:20:0 function g(a) { return f(...a); }
typein:20:0 .........................^

But I believe it is coming soon — bug 762363 is the one to watch, patch, and test.

for-of iteration

I blogged and spoke about for-of at TXJS 2011. The of contextual keyword, also in CoffeeScript, goes where in goes in for-in loops, in order to trigger the new iteration protocol (which is based on Python’s).


js> for (var v of [1, 2, 3]) print(v)
1
2
3

Arrays are iterable out of the box in ES6. This is a huge usability win! Unwary newcomers hoping for Pythonic value iteration can now avoid the trap of for-in on arrays iterating string-coerced keys rather than values.

Objects are not iterable without the programmer opting in explicitly:


js> for (var [k, v] of {p: 3, q: 4, r: 5}) print(k, v)
typein:24:0 TypeError: ({p:3, q:4, r:5}) is not iterable

To opt in, call an iterator factory, that is, a function that returns a fresh iterator for its parameter. Or simply give your objects or their common prototype an iterator property whose value is an iterator factory method: a function that returns the desired fresh iterator given its this parameter.

We require opt-in to avoid future-hostility against custom iterators for collection objects. Such objects probably do not want any kind of general property iterator default, which if left on Object.prototype, might be object-detected and prevent installation of the correct custom iterator factory.

The easiest way to create such an iterator factory is to write a generator function:


js> function items(o) { for (var k in o) yield [k, o[k]]; }
js> for (var [k, v] of items({p: 3, q: 4, r: 5})) print(k, v)
p 3
q 4
r 5

(This example uses destructuring, too.)

Note that SpiderMonkey has not yet implemented the ES6 generator function* syntax. We also haven’t added the ES6 features of delegating to a sub-generator via yield* and of returning a value from a generator (as in PEP 380). We’ll get to these soon.

Map

Have you ever wanted to map from arbitrary keys to values, without having the keys be implicitly converted to strings and therefore possibly colliding? ES6 Map is for you:


js> var objkey1 = {toString: function(){return "objkey1"}}
js> var objkey2 = {toString: function(){return "objkey2"}}
js> var map = Map([[objkey1, 42], [objkey2, true]])
js> map.get(objkey1)
42
js> map.get(objkey2)
true

The Map constructor takes any iterable, not just an array, and iterates over its key/value array-pairs.

Of course you can update a Map entry’s value:


js> map.set(objkey1, 43)
js> map.get(objkey1)
43

And you can add new entries with arbitrary key and value types:


js> map.set("stringkey", "44!")
js> for (var [k, v] of map) print(k, v)
objkey1 43
objkey2 true
stringkey 44!
js> map.size()
3

You can even use a key as a value:


js> map.set(objkey2, objkey1)
js> map.set(objkey1, objkey2)
js> for (var [k, v] of map) print(k, v)
objkey1 objkey2
objkey2 objkey1
hi 44
stringkey 44!

but now there’s a cycle between the objkey1 and objkey2 entries. This will tie up space in the table that must be manually released by breaking the cycle (or by dropping all references to the map):


js> map.delete(objkey1)
true
js> map.delete(objkey2)
true
js> for (var [k, v] of map) print(k, v)
hi 44
stringkey 44!

Setting the objkey1 and objkey2 variables to null is not enough to free the space in map tied up by the cycle. You must map.delete.

If your map is not exposed via an API by which arbitrary values could be passed as key and value to map.set, you won’t have to worry about cycles. And if the map itself becomes garbage soon (for sure), no worries. But for leak-proofing with arbitrary key/value cycles, see WeakMap, below.

Set

When you just want a set of arbitrary values, it’s a pain to have to use a map and burn code and memory on useless true values for the keys. So ES6 also offers Set:


js> var set = Set([1, true, "three"])
js> set.has(1)
true
js> set.has(2)
false
js> for (var e of set) print(e)
1
true
three
js> set.size()
3

As with Map, with a Set you can delete as well as add:


js> set.delete("three")
true
js> for (var e of set) print(e)
1
true
js> set.size()
2
js> set.add("three")
js> set.size()
3

An object element keyed by its identity works just as well as any other type of element.


js> var four = {toString: function(){return '4!'}}
js> set.add(four)
js> set.has(four)
true
js> for (var e of set) print(e)
1
true
three
4!

Unlike Map there is no cyclic leak hazard with arbitrary elements, although a WeakSet taking only object elements would still be helpful for automatic element removal when no other references to an element object remain. This idea has come up in connection with proxies and symbols, but I’ll save that for another post.

WeakMap

As noted above, with Map, making a cycle among map keys and values can tie up space in the table, and in the heap in all objects linked along the cycle or reachable from those objects, even when no references outside of the table to the key objects still live. Non-object keys, which can be recreated (forged) by writing literal string-equated expressions, have no such hazard.

ES6 WeakMap rides to the rescue:


js> var wm = WeakMap()
js> wm.set(objkey1, objkey2)
js> wm.set(objkey2, objkey1)
js> wm.has(objkey1)
true
js> wm.get(objkey1)
({toString:(function (){return "objkey2"})})
js> wm.has(objkey2)
true
js> wm.get(objkey2)
({toString:(function () {return 'objkey1'})})

So far so good, wm has a cycle but the objkey1 and objkey2 variables still keep the objects alive. Let’s cut the external references and force garbage collection:


js> objkey1 = null
null
js> gc()
"before 286720, after 282720\n"
js> wm.get(objkey2)
({toString:(function () {return 'objkey1'})})
js> objkey2 = null
null
js> gc()
"before 286720, after 282624\n"

At this point wm is empty. We can’t tell, however: there’s no way to enumerate a WeakMap, as doing so could expose the GC schedule (in browsers, you can’t call gc() to force a collection). Nor can we use wm.has to probe for entries, since we have nulled our objkey references!

A WeakMap is therefore close friends with the JS garbage collector. The GC knows when no references to a key object survive, and can collect the entry for that key — and for any cyclic entries in the table tied in a knot by their values being keys of other entries.

This special GC handling adds overhead, which ordinary Map users should not have to suffer.

What’s more, WeakMap accepts only object keys to enforce the no-forged-key rule necessary for the GC to be able to collect entries whose keys no longer survive — otherwise when could you ever GC an entry for key "if", which is typically interned along with the other JS reserved identifiers forever?

An entry with a key such as 42 or "42!" might be GC’ed if no copies of the key’s primitive value exist, even though the value could be recreated at any time (primitive types have value identity, not reference identity).

Of course, the GC cannot keep count of live instances of 42 very efficiently — or at all — depending on the JS engine’s implementation details. And strings are not observably shared via references and therefore counted, either (small ones could be copied, and are in many engines).

This is all a bit of a brain bender, and probably more than the average Map user needs to know, but the need for WeakMap compared to separate weak reference (on the ES7 radar!) and Map facilities is real. Smalltalkers discovered it decades ago, and called the weak key/value pair an Ephemeron (note: @awbjs, who witnessed the discovery, testified to me that the wikipedia page’s credits are incomplete).

Proxy

The draft ES6 spec has evolved since Proxies were first prototyped, but the good news is that the new Proxy spec can be implemented on the old one (which was prototyped in SpiderMonkey and V8) via Tom Van Cutsem‘s harmony-reflect library. The even better news is that the built-in direct proxies implementation has just landed in SpiderMonkey.

Tom’s __noSuchMethod__ implementation using direct proxies:


js> var MethodSink = Proxy({}, {
  has: function(target, name) { return true; },
  get: function(target, name, receiver) {
    if (name in Object.prototype) {
      return Object.prototype[name];
    }
    return function(...args) {
      return receiver.__noSuchMethod__(name, args);
    }
  }
});
js> void Object.defineProperty(Object.prototype,
  '__noSuchMethod__',
  {configurable: true, writable: true, value: function(name, args) {
    throw new TypeError(name + " is not a function");
  }});
js> var obj = { foo: 1 };
js> obj.__proto__ = MethodSink;
({})
js> obj.__noSuchMethod__ = function(name, args) { return name; };
(function (name, args) { return name; })
js> obj.foo
1
js> obj.bar()
"bar"
js> obj.toString
function toString() {
    [native code]
}

With this approach, you have to insert MethodSink just before the end of the prototype chain of an object that wants __noSuchMethod__‘s magic, using the __proto__ de facto standard that will be a de jure standard in ES6. The Object.prototype.__noSuchMethod__ backstop throws to catch bugs where the MethodSink was not on a receiver’s prototype chain.

This implementation does not just call the __noSuchMethod__ hook when a missing method is invoked, as shown after the obj.bar() line above. It also creates a thunk for any get of a property not in the target object and not in Object.prototype:


js> obj.bar
(function (...args) {
      return receiver.__noSuchMethod__(name, args);
    })
js> var thunk = obj.bar
js> thunk()
"bar"

I think this is an improvement on my original __noSuchMethod__ creation all those years ago in SpiderMonkey.

(Avid SpiderMonkey fans will cheer the switch to source recovery from decompilation evident in the result from Function.prototype.toString when evaluating obj.bar, thanks to Benjamin Peterson’s fine work in bug 761723.)

RegExp sticky (y) flag

This flag causes its regular expression to match in the target string starting from the index held in the lastIndex property of the regexp. Thus ^ can match at other than the first character in the target string. This avoids O(n2) complexity when lexing a string using a regexp, where without y one would have to take successive tail slices of the string and match at index 0.

String startsWith, endsWith, contains

These explain themselves by their names and they’re pretty simple, but also handier and more readable than the equivalent indexOf and lastIndexOf expansions.

Number isNaN, isFinite, toInteger, isInteger

The first two are not super-exciting, but worthwhile to avoid implicit conversion mistakes in specifying the isNaN and isFinite global functions which date from ES1 days:


js> Number.isNaN("foo")
false
js> isNaN("foo")
true

True fact: isNaN(" ") returns false because a string containing spaces converts (I was influenced by Perl; hey, it was the ’90s!) to the number 0, which sure enough is not a NaN. Dave Herman used this to good effect in the fun bonus segment of his BrazilJS talk.

The Integer static methods also avoid implicitly converting non-numeric arguments (e.g., "foo" to NaN). Their main purpose is to provide built-in IEEE-754 integral-double handling:


js> Math.pow(2,53)/3
3002399751580330.5
js> Number.isInteger(Math.pow(2,53)/3)
false
js> Math.pow(2,54)/3
6004799503160661
js> Number.isInteger(Math.pow(2,54)/3)
true

Notice how once you exceed the bits in IEEE double’s mantissa, Number.isInteger may return true for what you might wish were precise floating point results. Better luck in ES7 with value objects, which would enable new numeric types including IEEE-754r decimal.

Older prototype implementations

SpiderMonkey of course supports many Harmony proposals implemented based on ES4 or older drafts, e.g., const, let, generators, and destructuring. These will be brought up to spec as we hammer out ES6 on the anvil of Ecma TC39, heated in the forge of es-discuss, and user-tested in Firefox, Chrome, and other leading browsers. I hope to blog soon about other progress on the ES6 and ES7 “Harmony” fronts. A few clues can be found near the end of my Strange Loop slides.

/be

PS: I colorized the code examples in this post using the fun Prism.js library adapted lightly to ES6. Many thanks to Lea Verou, et al., for Prism.js!

18 Jun 2012

Recent talks: Fluent, TXJS 2012

I gave two talks recently, first at O’Reilly Media’s go-big-with-JavaScript FluentConf, and then at my favorite regional JS conference, the delightful TXJS (gorgeous site design), curated and stage-managed by Alex Sexton, Rebecca Murphey, and other usual suspects.

My Fluent video was up in record time, one achievement that the O’Reilly folks can brag about:

There I played the JS clip from Gary Bernhardt‘s hilarious Wat lightning talk, and gave bleeding-edge demos that my TXJS talk updated (video link for TXJS to be posted as soon as it’s available).

At TXJS, my title perhaps referenced Larry Niven (but let’s hope not), and my content directly cited the work of the late Lynn Margulis, champion of endosymbiotic theory. If JS is a mitochondrion, what in our modern web world correspond to the eukarya? I suspect we are in the midst of finding out. Node is probably involved.

At TXJS I mixed new metaphors like mixed drinks, harkened back to my last year’s TXJS talk, and gave the latest demos.

The big-picture story is renewed humility in the face of community feedback. My goal is to help the JS standards body be the best it can be as one of several symbiotic critters in an evolving system.

In last year’s TXJS talk I elaborated on how Ecma TC39 works, and also malfunctions sometimes. The important point is the hermeneutic spiral.

The spiral lives, and it works — although newcomers to es-discuss sometimes think consensus has been achieved within one 16-message thread. Not so fast! But consensus on ES6 is being achieved.

Lots of learning and re-learning here:

  • Versioning is an anti-pattern on the web, wherefore 1JS.
  • Syntax as programming language user-interface should evolve, but unlike standard library additions, developers can’t fix it or abstract over it (no macros yet, sigh). I made JS’s object model mutable at the start so people could polyfill, and they continue to do so for good reason. New syntax has to be minimal, righteous, and user-tested to get into Harmony, and that’s the way it should be.
  • One new syntax-suite that many agree pays its way, maximally minimal classes, still isn’t in ES6. Working on it…
  • My old “dunder-proto” (LOL, @littlecalculist was inspired in this pronunciation of __proto__ by Dunder-Mifflin) vanquished triangle (ugly typography didn’t help triangle either).
  • People still rant about JS’s privileged client-side status, but it’s a curse more than a blessing (Python, Lua, Ruby all would have been frozen badly if wedged into Netscape 2 in 1995).

The only hope is mutualism in an evolutionary struggle toward something better than either TC39 or JS developers could achieve by themselves. I tend to believe that this struggle will end well, better than master-planned would-be alternatives that don’t have a prayer of catching on as native-VM-implemented-yet-cross-browser on the Web as currently constituted.

JSFixed represents both a cry from the heart by some in the JS developer community, and an attractor for bikeshedding and tons of noise. But with excellent curation by @angustweets, @valueof, @rwaldron, and @KitCambridge, it has produced a reasonable set of recommendations. Now it’s up to TC39 to engulf the recommendations that it can stomach, as if we were a hungry cell and they a primitive bacterium, and then for both sides to find mutual wins in the resulting ensemble.

These are straightforward, but I expanded on maximin classes in the next slide.

Only => (fat arrow) made it into ES6; thin arrow may be an arrow too far, but I will try again at the July TC39 meeting.

The existential operator, ?., is relatively straightforward. I will put it on the agenda for July. It could fit in ES6, IMHO, without breaking any budgets or agreements.

I intend to write up Object.prototype.forEach as a strawman, based on JSFixed’s recommendation. It relies on Object.keys order, which in turn is for-in order, but no big deal. It’s a useful object method, shadowed by Array.prototype.forEach.

JS in 2gyr? More like at birth + 35yr. CoffeeScript forks off and rejoins with Ruby bacterial DNA engulfed. Is that Dart near the bottom? :-P

Evolution does sometimes paint a clade into a dead-end corner, or leave the evolved system with harsh dilemmas and “pick any two” trilemmas.

The quest for shorter function syntax runs afoul of this binding. As I said in my talk, if we are counting fairly, then => and -> are not a single short function syntax, they are two syntaxes grouped by being “arrows” or having two chars ending in >.

This slide refers to the default operator proposal for Harmony, which I’ve recently edited based on several rounds of es-discuss and twitter feedback. It is looking good, IMHO, with the only remaining issue (also open for parameter default values) of whether null as well as undefined should trigger defaulting.

Somehow, @rmurphey knew I would be speaking about Unicorns.

ES6 is already partly implemented in top browsers, and it is coming to more browsers soon. Time to start experimenting with it and giving feedback.

My demos:

So yeah: 3D games, Flash, and C/C++ in JS. Can you dig it?

I close by praising Wesley Snipes again (this time with a jwz memory), and taunting those who doubt either JS or Passenger 57.

I do not taunt for the sake of JS in itself, which started life as good-not-great where it wasn’t just silly. Rather, for JS as an over-maligned, unique evolving system that somehow still unites a standards group (made of competing browser vendors) with faithful JS developers, working together for a better symbiosis.

No other language has the curse or the blessing of this fate. Let’s do our best. Thanks to the JSFixed crew for doing theirs.

/be

15 Apr 2012

The infernal semicolon

Most of the comments in this semicolons in JS exchange make me sad. The code in question:

  clearMenus()
  !isActive && $parent.toggleClass('open')

relies on Automatic Semicolon Insertion (ASI) and so cannot be minified except by parsing fully (including ASI), observing the significance of the newline after clearMenus(), and inserting a semicolon when stripping that newline.

Some argue that JSMin has a bug. Doug Crockford does not want to change JSMin, and that’s his choice.

FWIW, I agree with Doug’s canonically grumpy tone if not his substance; more below on the substance.

I also agree with @cramforce and @jedschmidt that the && line is an abusage, allowed due to JS’s C heritage by way of Java, but frowned upon by most JS hackers; and that an if statement would be much better style (and, I take it, help JSMin do right). But this particular criticism is too ad hoc to help resolve the general “Let me have my ASI freedom and still minify, dammit!” debate.

Doug goes on to say:

TC39 is considering the use of ! as an infix operator. This code will break in the future. Fix it now. Learn to use semicolons properly. ! is not intended to be a statement separator. ; is.

The !-as-infix-operator idea is proposed as syntactic sugar for promises, which may or may not make it into Harmony with that exact syntax, or with any syntactic sugar at all.

Doug’s right that ! is not a statement terminator or “initiator”. And (my point here), neither is newline.

But search for [nlth] in the proposed promises grammar and you’ll see something surprising about ASI and infix operators: we can add new infix operators in the future, whether new contextual keyword-operators (e.g., is and isnt — BTW these are in doubt) or retasked, existing unary-prefix operators, provided that we insist on [no LineTerminator here] immediately to the left of any such infix operator.

(In ECMA-262, [no LineTerminator here] is used in so-called “restricted productions” to make contextually-significant newlines, e.g., after return without any expression of the return value on the same line.)

This future-friendliness to new infix operators comes directly from ASI as a newline-sensitive error correction procedure, as the example at top demonstrates. Try other examples using a leading identifier on a well-formed second line and you’ll see the same effect. Removing the newline introduces an early error, which creates homesteading space for new infix operators in a later edition of ECMA-262. Examples:

let flag = x is y;  // no \n before 'is'!
x ! p = v;          // Q(x).put(’p’, v)

An aside on coding style: if we add new infix operators used in restricted productions, this gives weight to the JS coding style that puts infix operators in multiline expressions at the end of continued lines, rather than at the beginning of continuation lines.

So while I agree with Doug on those two lines of code from Bootstrap (an excellent JS library, BTW) exhibiting poor style, it is not the case that such code as written could break in the future, even if we were to adopt the !-as-infix-operator strawman. The first line terminator in that example is indeed significant.

The moral of this story: ASI is (formally speaking) a syntactic error correction procedure. If you start to code as if it were a universal significant-newline rule, you will get into trouble. A classic example from ECMA-262:

a = b + c
(d + e).print()

Similar hazards arise with [, /, and unary + and -. Remember, if there wasn’t an error, ASI does not apply.

This problem may seem minor, but JS file concatenation ups the ante. For this reason some style guides (Dojo, IIRC) advocate starting your reusable JS file with ;, but people don’t know and it’s easy to forget.

I wish I had made newlines more significant in JS back in those ten days in May, 1995. Then instead of ASI, we would be cursing the need to use infix operators at the ends of continued lines, or perhaps \ or brute-force parentheses, to force continuation onto a successive line. But that ship sailed almost 17 years ago.

The way systematic newline significance could come to JS is via an evolution of paren-free that makes it to Harmony status. I intend to work on this in the strawman, but not for ES6.

Some of the github issue comments are naive or idealistic to the point of being silly. Since when does any programming language not have syntax arguments? All living, practical languages that I know of, even those with indentation-based block structure and similar restrictions, have degrees of freedom of expression that allow abusage as well as good usage. Language designers can try to reduce degrees of freedom, but not eliminate them completely.

My two cents: be careful not to use ASI as if it gave JS significant newlines. And please don’t abuse && and || where the mighty if statement serves better.

I’ll also say that if it were up to me, in view of JS’s subtle and long history, I’d fix JSMin. But I would still log a grumpy comment or two first!

/be

22 Feb 2012

Mobile Web API Evolution

Ragavan Srinivasan’s post about the forthcoming Mozilla Marketplace for Open Web Apps inspired me to write about Mozilla’s surging Web and Device API standards work.

A bit of background. Mozilla has always contributed to web standards, going back to the start of the project. We co-founded the WHAT-WG to kick off HTML5. As readers of this blog know, we are a leader in JS standardization. We have some of the top CSS and layout experts in the world.

In the last eight months, our efforts to extend the web standards to include new APIs needed to build compelling apps and OS components on mobile devices have really caught fire. B2G and Open Web Apps are the fuel for this fire.

So I thought I would compile a list of emerging APIs to which we’ve contributed. In citing Mozillans I do not mean to minimize the efforts of standardization colleagues at Google, Microsoft, Nokia, Opera, the W3C and elsewhere. Standards are a multi-vendor effort (although excluding WebGL [see UPDATE below] one shiny name is conspicuously absent from this list).

The Mozilla contributions are worth noting both to acknowledge the individuals involved, and to highlight how Mozilla is championing device APIs for the web without having a native application stack blessed with such APIs on offer. We see the Web as quickly evolving to match native stacks. We have no other agenda than improving the Web to improve its users’ lives, including Web developers’ lives — especially mobile users and developers.

As always, standards in progress are subject to change, yet require prototype implementation and user-testing. Mozilla remains committed to playing fairly by not forging de-facto standards out of prototypes, rather proposing before disposing and in the end tracking whatever is standardized.

Here is the list, starting with some 2011-era work:

  • Geolocation, with Google contributing the editor and Firefox (thanks to Jay Sullivan leading the charge) implementing early.
  • WebGL (UPDATE: Chris Marrin of Apple edited) and typed arrays.
  • Gamepad API. Co-editor: Ted Mielczarek. Mozillans are also contributing to Pointer Lock.
  • Screen Orientation. Editor: Mounir Lamouri.
  • navigator.getUserMedia. Co-editor: Anant Narayanan
  • Battery Status (in Last Call). From the Acknowledgements:

    Big thanks to the Mozilla WebAPI team for their invaluable feedback based on prototype implementations.

  • Media Capture. Fabrice Desré prototype-implemented in Gecko.
  • Network API. Editor: Mounir Lamouri.
  • Web Telephony. Ben Turner, Jonas Sicking, Philipp von Weitershausen.
  • Web SMS. Mounir Lamouri, Jonas Sicking.
  • Vibration. From the Acknowledgements:

    The group is deeply indebted to Mounir Lamouri, Jonas Sicking, and the Mozilla WebAPI team in general for providing the WebVibrator prototype as an initial input.

  • File API. Editors: Arun Ranganathan, Jonas Sicking.
  • IndexedDB. Editors includes Jonas Sicking.

I did not list most of the HTML5 and Web API work aimed at Desktop Firefox, to focus on the new mobile-oriented additions. There’s more to say, including about bundled-permission follies and how to weave permission-granting (with memorization) into interactions, but not here.

One last note. The CSS vendor prefix brouhaha had, among many salutary effects, the benefit of shining light on an important requirement of competitive mobile web development: CSS style properties such as -webkit-animation-*, however you spell them, must have fast and beautiful implementations across devices for developers to find them usable: 60Hz, artifact-free rendering under touch control. This requires such work as off-main-thread compositing and GL layers.

This is a high technical bar, but we are in the process of meeting it in the latest Firefox for Android and B2G builds, thanks to hard work from many people, especially Patrick Walton, Robert O’Callahan, Chris Jones, and Andreas Gal. Onward!

/be

28 Oct 2011

JSConf.eu

JSConf.eu 2011 was terrific, bigger and juicier than last year, with a strong sense of community felt from reject.js pre-conf:

to start:

to finish:

Chris Williams makes a moving plea for an end to negativity, meaning trolling, flaming, mocking, and hating in online media.

This sounds utopian, like “an end to history”. But it is good as an aspiration, a constant reminder, since we’ve all seen how many people tend to be more negative online than they are in person. This isn’t just a matter of isolated individual behavior, free of cultural feedback loops. The new media reinforce tribalism.

However, it is hard to be positive about some things. I will persevere….

JSConf.eu had too many awesome talks to cover without bi-locating. Mozillans were well-represented, including dmandelin and dvander on JavaScript JITs, Marijn Haverbeke on DOM implementation techniques, Chris Heilmann on Community JS reloaded – how to rock as a movement, and Andreas Gal on PDF.js. Janet Swisher led the MDC doc sprint in the Hacker Lounge.

I would like to single out Alon Zakai‘s Emscripten talk. Emscripten is an LLVM-to-JS compiler, which means it enables compiling C, C++, and Objective-C (and other languages with LLVM front ends) to JS. What’s more, interpreters written in C for Python, Ruby, and Lua have been compiled and hosted on the web.

Alon’s results are impressive, with lots of room for more wins. At JSConf.eu, jaws dropped and eyes were opened.

For my talk, I reprised some CapitolJS material, including the RiverTrail demo, which won loud and enthusiastic applause when I clicked on the “Parallel” button.

(A few people asked afterward about whether the graphics was running on one of four cores. I’ll repeat the answer here: the particle system demo uses WebGL targeting the GPU for rendering, and the four CPUs’ vector units for n-body solving. All from deadlock-free, data-race-free, seemingly single-threaded JS.)

Here’s the video of my talk:

The amazing Anna Lena Schiller created infographics for all the talks, on the spot — a truly impressive display of concentration and stamina. Here’s the one she did for my talk:

And here are the updated and new slides I presented, showing ES6 work-in-progress (none of it final, so don’t panic) and covering some current controversies.

From recent es-discuss messages, I’m afraid that classes are on their way out of ES6. This seems a shame, and avoidable. In hindsight, we did not have all class advocates working in concert on the hard issues last year and earlier this year. But we also do not agree on what’s required for ES6, and some on TC39 view minimizing as future-hostile.

To be blunt, we lost some “classes” advocates who work for Google to Dart. Others at Google on TC39 seem to want more out of ES6 classes than even Dart guarantees (see the future-hostile point above).

I’m not slamming Google as a company here, since it does still support people working on JS in TC39. I respect the people involved and believe they’re for the most part making their own choices. But Dart and other unrelated Google agenda items do impose clear and significant opportunity costs on Google’s standards actiivities.

To remain positive per “An End to Negativity”, I’ll simply conclude that we TC39ers should pay attention to Dart now that it is out, even though we’ve lost time and potential contributions.

The famous Tony Hoare quote that Bill Frantz cited, which argues for deferring classes, is this:

When any new language design project is nearing completion, there is always a mad rush to get new features added before standardization. The rush is mad indeed, because it leads into a trap from which there is no escape. A feature which is omitted can always be added later, when its design and its implications are well understood. A feature which is included before it is fully understood can never be removed later.
From C.A.R.Hoare’s 1980 ACM Turing Award Lecture

I agree with Erik Arvidsson that “[b]y not providing [class] syntax we are continuing to encourage a million incompatible ‘class’ libraries.” I’m with Erik: I would still like to see TC39 agree on minimal classes. But not at any cost.

Onward to new proposals with sometimes-tentative syntax. I’m continuing to “live in a fishbowl” by showing these proposals, even though doing so risks drive-by misinterpretation that we have finalized the sum of all proposals.

So, please don’t freak out. Not all of this will make it as proposed. We may also make cuts. But it’s important to address the use-cases motivating these proposals, take in the fullness of the problem space and potential solutions, and do the hermeneutic spiral.

Apart from font issues that make <| look lopsided or non-triangular, this proposal looks good. It replaces the main legitimate use-case for assigning to __proto__: presetting the prototype link in an object literal.

Unlike Object.extend, .{ copies only “own” properties from its right-hand-side object literal, and (this is a crucial difference) it also copies properties with private name object keys (which are non-enumerable by definition). For example, base.{[privateName]: value, publicName: value2} given a private name object reference denoted privateName in scope.

Design patterns point to programming language bugs. Nevertheless, this class pattern shows clever work by Allen Wirfs-Brock, decomposing classes-as-sugar into chained operator expressions. It’s still a bit verbose and error-prone in my opinion, and cries out for the ultimate sugar of minimal class syntax (if only we could agree on that).

Much of the Dart class syntax design looks good to me. Possibly TC39 can agree to adopt it, with necessary adjustments. It would still be sugar for constructors and prototypes.

Arrow function syntax faces an uphill battle due to the combination of TC39′s agreement to future-proof by having an unambiguous LR(1) grammar (after ASI and with lookahead restrictions); mixed with the comma expression, (a, b, c), which I copied into JS’s grammar straight from C (not from Java, which left it out, instead providing comma-separated special forms in a few contexts, e.g. for(;;) loop heads). You can’t have both, and we do not want to remove the comma expression in Harmony.

I’m quite in favor of block-lambdas, and they meet formal approval from TC39′s strictest grammarian. Some still object to them as an alien DNA injection from Ruby and Smalltalk, both syntactically and (with Tennent Correspondence Principle conformance regarding return, break, continue, and this) semantically.

At this point, ES6 has no shorter function syntax. This seems like a loss, and fixable, to me. Your comments welcome, especially if they make novel distinctions that help forge consensus.

During the talk and Q&A, I recounted how the WHAT-WG was created to counteract a standards body gone wrong (the 2004-era W3C). I then raised the idea of a community-based group, a “JS-WG”, to augment the much healthier but still under-staffed Ecma TC39 committee.

Besides floating more ideas (really, the point is not to bikeshed endlessly or take in too many proposals to digest), a JS-WG worth organizing might actually develop draft specs and prototype implementation patches for JavaScriptCore, SpiderMonkey, and V8. The maintainers of those engines could use the help, and with patches and patched builds, we could scale up user testing beyond what’s in the cards now.

I know it’s hard to believe, but people are finally realizing that with V8 prototyping alongside SpiderMonkey, ES6 is happening. It’ll be prototyped in pieces. I hope many will be “on by default” (e.g., not under a flag in Chrome) well before the new edition is standardized (end of 2013). That’s how we roll in Firefox with SpiderMonkey.

/be

23 Sep 2011

CapitolJS, RiverTrail

I took time away from the Mozilla all-hands last week to help out on-stage at the Intel Developer Forum with the introduction of RiverTrail, Intel’s technology demonstrator for Parallel JS — JavaScript utilizing multicore (CPU) and ultimately graphics (GPU) parallel processing power, without shared memory threads (which suck).

Then over the weekend, I spoke at CapitolJS, talking about ES6 and Dart, and demo’ing RiverTrail to the JS faithful. As usual, I’ll narrate my slides, but look out for something new at the end: a screencast showing the RiverTrail IDF demo.

I had a lot to cover in a half-hour (a good talk-time in my view — we’ll see how the video comes out, but I found it invigorating). CapitolJS had a higher-than-JSConf level of newcomers to JS in attendance, so I ran through material that should be familiar to readers of this blog, presented here without much commentary:

The meaning of my color-coding should be intuitively obvious ;-).

BTW, dom.js, with its Proxy usage under the hood, is going great, with David Flanagan and Donovan Preston among others hacking away on it, and (this is important) providing feedback to WebIDL‘s editor, Cameron McCormack.

Here I must add the usual caveat that “ES6″ might be renumbered. Were I more prudent, I’d call it “ES.next”, but it’s highly likely to be the 6th edition, and you’re all sophisticated close readers of the spec and its colorful history, right?

The SpiderMonkey bug tracking binary data prototype implementation is bug 578700.

The SpiderMonkey bug tracking quasi-literal prototype implementation is bug 688857.

The @ notation is actually “out” for ES6, per the July meeting (see notes). Thanks to private name objects and object literal extensions (see middle column), private access syntax is factored out of classes. Just use this[x] or p[x], or (in an object literal for the computed property name, which need not be a private name) [x]: value.

As close to CoffeeScript as I can get them, given JS’s grammar and curly-brace (not indentation-based) block structure.

Ruby-esque, Smalltalk is the grandfather.

You know you want it.

I’ve been clear about preferring block-lambdas over arrows for their added semantic value and compelling syntax mimicking control statements. It’s good to hear @jashkenas agree in effect.

@mikeal is right that JS syntax is not a problem for some (perhaps many) users. For others, it’s a hardship. Adding block-lambdas helps that cohort, adds value for code generators (see Harmony Goals above), and on balance improves the language in my view. It won my straw show-of-hands poll at CapitolJS against all of arrows, vaguer alternatives, and doing nothing.

The epic Hacker News thread on my last blog post in relation to Dart needs its own Baedeker. For now I’ll just note that Dart and the politics evident in the memo are not making some of my standards pals who work for other browser vendors happy. Google may fancy itself the new Netscape, but it doesn’t have the market share to pull off proprietary power-move de facto standards.

The leaked memo makes some observations I agree with, some unbacked assertions about unfixable JS problems that TC39 work in progress may falsify this year, and a few implicit arguments that are just silly on their face.

Still, I think we should react to valid complaints about JS, whatever the source. The number type has well-known usability and (in practice, in spite of aggressively optimizing JIT-compiling VMs) performance problems.

The last bullet shows pragmas for switching default numeric type and arithmetic evaluation regime. This would have to affect Number and Math, but lexically — no dynamic scope. Still a bit hairy, and not yet on the boards for Harmony. But perhaps it ought to be.

Links: [it's true], [SpiderMonkey Type Inference].

Coordinated strawman prototyping in SpiderMonkey and V8 is a tall order. Perhaps we need a separate jswg.org, as whatwg.org is to the w3c, to run ahead? I’ve been told I should be BDFL of such an org. Would this work? Comments welcome.

Remember, ridiculously parallel processing power is coming, if not already present, on your portable devices. It’s here on your laptops and desktops. The good news is that JS can exploit it without your having to deal with data races and deadlocks.

RiverTrail is a Narcissus-based JS to OpenCL compiler, packaged as a Firefox add-on. It demonstrates the utility of a new ParallelArray built-in, based on typed arrays. The JS-to-OpenCL compiler automatically multicore-short-vectorizes your JS for you.

As noted in the previous slide, because the ParallelArray methods compile to parallelized folds (see Guy Steele’s excellent ICFP 2009 keynote), associative operations will be reordered, resulting in small non-deterministic floating point imprecision errors. This won’t matter for graphics code in general, and it’s an inevitable cost of business using parallel floating point hardware.

The code looks like typical JS, with no hairy callbacks, workers, or threads. It requires thinking in terms of immutable trees and reductions or other folds, but that is not a huge burden. As Guy’s talk makes plain, learning to program this way is the key to parallel speedups.

Here is my screencast of the demo. Alas, since RiverTrail currently targets the CPU and its short vector unit (SSE4), and my screencast software uses the same parallel hardware, the frame rate is not what it should be. But not to worry, we’re working on GPU targeting too.

At CapitolJS and without ScreenFlow running, I saw frame rates above 35 for the Parallel demo, compared to 3 or 2 for Sequential.

The point of a technology demonstrator is to show where real JS engines can go. Automatic parallelization of ParallelArray-based code can be done by next year’s JS engines, based on this year’s Firefox add-on. We’re very close to exploiting massively parallel hardware from JS, without having to write WebCL and risk terrible safety and DoS bugs.

To close, I sought inspiration from Wesley Snipes in Passenger 57. Ok, not his best movie, but I miss the ’90s action movie era.

Seriously, the shortest path on the Web usually is the winning play. JS is demonstrably able to grow new capabilities with less effort than a “replacement” entails. Always bet on JS!

/be

24 Aug 2011

My TXJS talk (Twitter remix)

TXJS 2011 A6 – Brendan Eich – Ecma TC39: The Good, The Bad, and The Ugly.

[Main slides] [Paren-free]

I spoke at TXJS, a really excellent regional JS conference, in June. Thanks to @SlexAxton, rmurphey, and everyone else involved. My talk was concerned with the good, bad, and ugly of Ecma TC39 (and I mean those words in the best possible way!), mixing philosophy with historical events from the last 14 years.

After describing the standards process and its history in Ecma, I presented the good stuff that’s going into ES6 (I gave a remixed version of this part of the talk at an SFTechTalk hosted by Twitter last month — thanks to @valueof and @chanian for hosting). As a bonus, I closed with a paren-free update, whose punchline slide is included at the bottom.

I am not Tuco.

We don’t smoke, but the rest is pretty much right.

Here I praised Jan van den Beld, former S-G, Ecma — a raconteur and polymath, for his stewardship of Ecma.

Yes, Netscape picked ECMA (now Ecma, pay attention ;-)) as standards body to which to submit JS mainly to give Microsoft grief, since ECMA had bravely standardized some part of the Windows API. But Jan and the entire TC39 TG1 group played fair on ES1, the first ECMA-262 Edition. Microsoft was so happy they standardized early C# and CLI metadata specs at Ecma.

Not Bruce Campbell, so not me. That means I’m the Bad.

Here is something that the Google leak about Dart (née Dash) telegraphs: many Googlers, especially V8 principals, do not like JS and don’t believe it can evolve “in time” (whatever that might mean — and Google of course influences JS’s evolution directly, so they can put a finger on the scale here).

They’re wrong, and I’m glad that at least some of the folks at Google working in TC39 actually believe in JS — specifically its ability to evolve soon enough and well enough to enable both more predictable performance and programming in the large.

There’s a better-is-better bias among Googlers, but the Web is a brutal, shortest-path, Worse-is-Better evolving system.

I’ve spent the last 16 years betting on the Web. Evolving systems can face collapses, die-offs, exigent circumstances. I don’t see JS under imminent threat of death due to such factors, though. Ironic that Google would put a death mark on it.

Here I propose that Crock is Plato and I am Aristotle, and that while we need to “keep it real”, we must also hew to high ideals.

The ideas I cite here, represented by the Hermenuetic Circle, definitely apply to TC39′s understanding of the text of ECMA-262, as well as various canonical texts in Computer Science. The committee works best when it spirals in on a solid design, avoiding local and premature optimizations and pessimizations.

Every one of these “Bad Parts” has been on parade in TC39 in recent years, including 2011. I’ve been tempted by the second one, horse-trading, so I’m not innocent (no one is).

The “Scenario Solving” one is particularly subtle in that the Scenario proposed to be solved is quite often a very real developer problem. But a complex, ad-hoc solution to it, especially when rushed, too often is unsound. We would do better to take the Scheme lesson to heart, and develop sound and valid orthogonal primitives. Higher-level libraries built on them can be standardized post hoc.

These meta-discussions are sometimes quite funny in light of the vendor whose representative is making them. I note that C# can now be written to look like JS. Why shouldn’t any particular extension to C# be at least considered (not rubber-stamped of course) for JS standardization?

These slides provide a glimpse of ES6, still under construction but already full of good stuff.

The Harmony Goals are not just lofty abstractions.

I hope the way JS is developed and standardized, as much in the open as the existing standards process permits, and with developer feedback via es-discuss, twitter, and the various conference talks, helps. If not, we may as well wait for some single-source solution to descend from the mountain. And then hold our breaths waiting for Firefox, IE and Safari to implement!

While paren-free in all likelihood won’t make ES6, the for-of loop will, and comprehensions and generator expressions in ES6 will be paren-free. Yay!

/be

5 May 2011

Mozilla’s NodeConf Presentation

NodeConf is a blast, and Mozilla had a 30 minute slot. Here’s the slideshare.net link.

SpiderNode and V8Monkey are on github, of course. Paul O’Shannessy already blogged a few weeks ago.

To avoid confusion, here’s the cheat-sheet:

  • V8Monkey is SpiderMonkey with V8′s API around it. We are not done emulating the full V8 API.
  • Because we haven’t managed to perfectly emulate the full V8 API, the few language-level extensions (e.g., Error.captureStackTrace), and the V8 build system, SpiderNode is a clone of Node with V8Monkey integrated and (in a few cases we want to get rid of) hacked in place of V8.

This slide should speak for itself.

We are not out to make a maintained, competing fork of Node, just a friendly downstream that should go away as soon as possible. We aren’t selling anything to Node users.

We are trying to improve SpiderMonkey’s API, test Harmony JS language features in the Node setting, and have fun learning about the new JS server side.

These four slides are straight from my JSConf.us 2011 talk. I went fast since a lot of NodeConf attendees were at JSConf, but a good number of hands did not go up when I asked who attended both conferences.

yield conquers the nested function spaghetti monster.

Generators are winning, and they are worth playing with and investigating in SpiderNode and of course Firefox and other SpiderMonkey embeddings (also in Rhino). They are not the last word on the “anti-function-nesting” topic, for sure. I’m pretty sure there is no “last word”.

Thanks to Rob Arnold for this demo and the next one, and of course to Dave Herman for TaskJS. Thanks too to Shawn Wilsher for the encouragement at the last minute ;-) .

This is an even shorter demo. I switched to a terminal window, fired up $ ./node helloworld.js, loaded http://127.0.0.1:10337/ into a fresh Firefox window, and pressed reload repeatedly to show the counter incrementing.

Node really did break the mold when it comes to ease of writing server code that you can get running super-fast, using JS and lots of the client-side knowledge you may already have.

One more time: thanks to @robarnold, @sdwilsh, @zpao, the awesome @john_h_ford who did our build automation, and of course my partner in crime for much mad science at Mozilla, @andreasgal.

Join us on IRC and the mailing list, we have a lot of work still to do, and we’re having a ton of fun.

/be

PromoteJS

JavaScript String .charAt

Tags

About Brendan

Brendan Eich co-founded mozilla.org and served as CEO for Mozilla. He is widely known for his contributions to the evolution of the Web, including inventing JavaScript and spearheading its ongoing standardization and evolution.