Harmony of Dreams Come True

This blog focuses on portions of the new-in-ES6 stuff I presented in my Strange Loop 2012 closing keynote, which was well-received (reveal.js-based HTML slides, some from my Fluent 2012 keynote, many of those originally from Dave Herman‘s Web Rebels 2012 talk [thanks!], can be viewed here; notes courtesy Jason Rudolph).

UPDATE: the Strange Loop keynote video is up.

---

I blogged early in 2011 about Harmony of My Dreams, to try to fit in one page some dream-sketches (if not sketchy dreams — the #-syntax ideas were sketchy) of what I thought were the crucial elements of ECMAScript Harmony, the name I coined for the standardized future of JavaScript.

Now this dream is coming true, not just in ES6 draft specs but in prototype implementations in top browsers. Here I’ll tout Firefox 15, which released almost six weeks ago (yes, this means Firefox 16 is tomorrow, and Firefox 17 beta and 18 aurora too — these all have yet more new goodies in them — isn’t Rapid Release fun?). Per the MDN docs, the SpiderMonkey JS engine shipped in Firefox 15 sports the following new prototype-implemented draft ES6 features:

Default parameters

This extension (AKA “parameter default values”) is too sweet, and it will help put the arguments object out to pasture:

js> function f(a = 0, b = a*a, c = b*a) { return [a, b, c]; }
js> f()
[0, 0, 0]
js> f(2)
[2, 4, 8]
js> f(2, 3)
[2, 3, 6]
js> f(2, 3, 4)
[2, 3, 4]

Implementation credit goes to Benjamin Peterson for his work implementing default parameters, and to Jason Orendorff for his always-excellent code reviews. See this bug for followup work to track the latest ES6 agreement on how passing undefined (and only undefined) should trigger defaulting.

We have a few details to iron out still about scope, I suspect (based on this es-discuss message and its thread).

Rest parameters

Even sweeter than default parameters are rest parameters, and I bet they are ahead of default parameters in making arguments a bad memory some fine day:

js> function f(a, b, ...r) { print(Array.isArray(r)); return r.concat(a, b); }
js> f(1, 2)
true
[1, 2]
js> f(1, 2, 3)
true
[3, 1, 2]
js> f(1, 2, 3, 4, 5)
true
[3, 4, 5, 1, 2]

Again credit goes to Benjamin and Jason for their work.

Spread in array literals

The dual of rest is called “spread”, and it should work in call expressions as well as array literals. The latter is implemented in Firefox 16 (now in the beta channel):

js> a = [3, 4, 5]
[3, 4, 5]
js> b = [1, 2, ...a]
[1, 2, 3, 4, 5]

Thanks once again to Benjamin (a star Mozilla intern this summer) and Jason.

Spread in call expressions is not yet implemented:

js> function f(...r) { return r; }
js> function g(a) { return f(...a); }
typein:20:0 SyntaxError: syntax error:
typein:20:0 function g(a) { return f(...a); }
typein:20:0 .........................^

But I believe it is coming soon — bug 762363 is the one to watch, patch, and test.

for-of iteration

I blogged and spoke about for-of at TXJS 2011. The of contextual keyword, also in CoffeeScript, goes where in goes in for-in loops, in order to trigger the new iteration protocol (which is based on Python’s).

js> for (var v of [1, 2, 3]) print(v)
1
2
3

Arrays are iterable out of the box in ES6. This is a huge usability win! Unwary newcomers hoping for Pythonic value iteration can now avoid the trap of for-in on arrays iterating string-coerced keys rather than values.

Objects are not iterable without the programmer opting in explicitly:

js> for (var [k, v] of {p: 3, q: 4, r: 5}) print(k, v)
typein:24:0 TypeError: ({p:3, q:4, r:5}) is not iterable

To opt in, call an iterator factory, that is, a function that returns a fresh iterator for its parameter. Or simply give your objects or their common prototype an iterator property whose value is an iterator factory method: a function that returns the desired fresh iterator given its this parameter.

We require opt-in to avoid future-hostility against custom iterators for collection objects. Such objects probably do not want any kind of general property iterator default, which if left on Object.prototype, might be object-detected and prevent installation of the correct custom iterator factory.

The easiest way to create such an iterator factory is to write a generator function:

js> function items(o) { for (var k in o) yield [k, o[k]]; }
js> for (var [k, v] of items({p: 3, q: 4, r: 5})) print(k, v)
p 3
q 4
r 5

(This example uses destructuring, too.)

Note that SpiderMonkey has not yet implemented the ES6 generator function* syntax. We also haven’t added the ES6 features of delegating to a sub-generator via yield* and of returning a value from a generator (as in PEP 380). We’ll get to these soon.

Map

Have you ever wanted to map from arbitrary keys to values, without having the keys be implicitly converted to strings and therefore possibly colliding? ES6 Map is for you:

js> var objkey1 = {toString: function(){return "objkey1"}}
js> var objkey2 = {toString: function(){return "objkey2"}}
js> var map = Map([[objkey1, 42], [objkey2, true]])
js> map.get(objkey1)
42
js> map.get(objkey2)
true

The Map constructor takes any iterable, not just an array, and iterates over its key/value array-pairs.

Of course you can update a Map entry’s value:

js> map.set(objkey1, 43)
js> map.get(objkey1)
43

And you can add new entries with arbitrary key and value types:

js> map.set("stringkey", "44!")
js> for (var [k, v] of map) print(k, v)
objkey1 43
objkey2 true
stringkey 44!
js> map.size()
3

You can even use a key as a value:

js> map.set(objkey2, objkey1)
js> map.set(objkey1, objkey2)
js> for (var [k, v] of map) print(k, v)
objkey1 objkey2
objkey2 objkey1
hi 44
stringkey 44!

but now there’s a cycle between the objkey1 and objkey2 entries. This will tie up space in the table that must be manually released by breaking the cycle (or by dropping all references to the map):

js> map.delete(objkey1)
true
js> map.delete(objkey2)
true
js> for (var [k, v] of map) print(k, v)
hi 44
stringkey 44!

Setting the objkey1 and objkey2 variables to null is not enough to free the space in map tied up by the cycle. You must map.delete.

If your map is not exposed via an API by which arbitrary values could be passed as key and value to map.set, you won’t have to worry about cycles. And if the map itself becomes garbage soon (for sure), no worries. But for leak-proofing with arbitrary key/value cycles, see WeakMap, below.

Set

When you just want a set of arbitrary values, it’s a pain to have to use a map and burn code and memory on useless true values for the keys. So ES6 also offers Set:

js> var set = Set([1, true, "three"])
js> set.has(1)
true
js> set.has(2)
false
js> for (var e of set) print(e)
1
true
three
js> set.size()
3

As with Map, with a Set you can delete as well as add:

js> set.delete("three")
true
js> for (var e of set) print(e)
1
true
js> set.size()
2
js> set.add("three")
js> set.size()
3

An object element keyed by its identity works just as well as any other type of element.

js> var four = {toString: function(){return '4!'}}
js> set.add(four)
js> set.has(four)
true
js> for (var e of set) print(e)
1
true
three
4!

Unlike Map there is no cyclic leak hazard with arbitrary elements, although a WeakSet taking only object elements would still be helpful for automatic element removal when no other references to an element object remain. This idea has come up in connection with proxies and symbols, but I’ll save that for another post.

WeakMap

As noted above, with Map, making a cycle among map keys and values can tie up space in the table, and in the heap in all objects linked along the cycle or reachable from those objects, even when no references outside of the table to the key objects still live. Non-object keys, which can be recreated (forged) by writing literal string-equated expressions, have no such hazard.

ES6 WeakMap rides to the rescue:

js> var wm = WeakMap()
js> wm.set(objkey1, objkey2)
js> wm.set(objkey2, objkey1)
js> wm.has(objkey1)
true
js> wm.get(objkey1)
({toString:(function (){return "objkey2"})})
js> wm.has(objkey2)
true
js> wm.get(objkey2)
({toString:(function () {return 'objkey1'})})

So far so good, wm has a cycle but the objkey1 and objkey2 variables still keep the objects alive. Let’s cut the external references and force garbage collection:

js> objkey1 = null
null
js> gc()
"before 286720, after 282720n"
js> wm.get(objkey2)
({toString:(function () {return 'objkey1'})})
js> objkey2 = null
null
js> gc()
"before 286720, after 282624n"

At this point wm is empty. We can’t tell, however: there’s no way to enumerate a WeakMap, as doing so could expose the GC schedule (in browsers, you can’t call gc() to force a collection). Nor can we use wm.has to probe for entries, since we have nulled our objkey references!

A WeakMap is therefore close friends with the JS garbage collector. The GC knows when no references to a key object survive, and can collect the entry for that key — and for any cyclic entries in the table tied in a knot by their values being keys of other entries.

This special GC handling adds overhead, which ordinary Map users should not have to suffer.

What’s more, WeakMap accepts only object keys to enforce the no-forged-key rule necessary for the GC to be able to collect entries whose keys no longer survive — otherwise when could you ever GC an entry for key "if", which is typically interned along with the other JS reserved identifiers forever?

An entry with a key such as 42 or "42!" might be GC’ed if no copies of the key’s primitive value exist, even though the value could be recreated at any time (primitive types have value identity, not reference identity).

Of course, the GC cannot keep count of live instances of 42 very efficiently — or at all — depending on the JS engine’s implementation details. And strings are not observably shared via references and therefore counted, either (small ones could be copied, and are in many engines).

This is all a bit of a brain bender, and probably more than the average Map user needs to know, but the need for WeakMap compared to separate weak reference (on the ES7 radar!) and Map facilities is real. Smalltalkers discovered it decades ago, and called the weak key/value pair an Ephemeron (note: @awbjs, who witnessed the discovery, testified to me that the wikipedia page’s credits are incomplete).

Proxy

The draft ES6 spec has evolved since Proxies were first prototyped, but the good news is that the new Proxy spec can be implemented on the old one (which was prototyped in SpiderMonkey and V8) via Tom Van Cutsem‘s harmony-reflect library. The even better news is that the built-in direct proxies implementation has just landed in SpiderMonkey.

Tom’s __noSuchMethod__ implementation using direct proxies:

js> var MethodSink = Proxy({}, {
  has: function(target, name) { return true; },
  get: function(target, name, receiver) {
    if (name in Object.prototype) {
      return Object.prototype[name];
    }
    return function(...args) {
      return receiver.__noSuchMethod__(name, args);
    }
  }
});
js> void Object.defineProperty(Object.prototype,
  '__noSuchMethod__',
  {configurable: true, writable: true, value: function(name, args) {
    throw new TypeError(name + " is not a function");
  }});
js> var obj = { foo: 1 };
js> obj.__proto__ = MethodSink;
({})
js> obj.__noSuchMethod__ = function(name, args) { return name; };
(function (name, args) { return name; })
js> obj.foo
1
js> obj.bar()
"bar"
js> obj.toString
function toString() {
    [native code]
}

With this approach, you have to insert MethodSink just before the end of the prototype chain of an object that wants __noSuchMethod__‘s magic, using the __proto__ de facto standard that will be a de jure standard in ES6. The Object.prototype.__noSuchMethod__ backstop throws to catch bugs where the MethodSink was not on a receiver’s prototype chain.

This implementation does not just call the __noSuchMethod__ hook when a missing method is invoked, as shown after the obj.bar() line above. It also creates a thunk for any get of a property not in the target object and not in Object.prototype:

js> obj.bar
(function (...args) {
      return receiver.__noSuchMethod__(name, args);
    })
js> var thunk = obj.bar
js> thunk()
"bar"

I think this is an improvement on my original __noSuchMethod__ creation all those years ago in SpiderMonkey.

(Avid SpiderMonkey fans will cheer the switch to source recovery from decompilation evident in the result from Function.prototype.toString when evaluating obj.bar, thanks to Benjamin Peterson’s fine work in bug 761723.)

RegExp sticky (y) flag

This flag causes its regular expression to match in the target string starting from the index held in the lastIndex property of the regexp. Thus ^ can match at other than the first character in the target string. This avoids O(n²) complexity when lexing a string using a regexp, where without y one would have to take successive tail slices of the string and match at index 0.

String startsWith, endsWith, contains

These explain themselves by their names and they’re pretty simple, but also handier and more readable than the equivalent indexOf and lastIndexOf expansions.

Number isNaN, isFinite, toInteger, isInteger

The first two are not super-exciting, but worthwhile to avoid implicit conversion mistakes in specifying the isNaN and isFinite global functions which date from ES1 days:

js> Number.isNaN("foo")
false
js> isNaN("foo")
true

True fact: isNaN(" ") returns false because a string containing spaces converts (I was influenced by Perl; hey, it was the ’90s!) to the number 0, which sure enough is not a NaN. Dave Herman used this to good effect in the fun bonus segment of his BrazilJS talk.

The Integer static methods also avoid implicitly converting non-numeric arguments (e.g., "foo" to NaN). Their main purpose is to provide built-in IEEE-754 integral-double handling:

js> Math.pow(2,53)/3
3002399751580330.5
js> Number.isInteger(Math.pow(2,53)/3)
false
js> Math.pow(2,54)/3
6004799503160661
js> Number.isInteger(Math.pow(2,54)/3)
true

Notice how once you exceed the bits in IEEE double’s mantissa, Number.isInteger may return true for what you might wish were precise floating point results. Better luck in ES7 with value objects, which would enable new numeric types including IEEE-754r decimal.

Older prototype implementations

SpiderMonkey of course supports many Harmony proposals implemented based on ES4 or older drafts, e.g., const, let, generators, and destructuring. These will be brought up to spec as we hammer out ES6 on the anvil of Ecma TC39, heated in the forge of es-discuss, and user-tested in Firefox, Chrome, and other leading browsers. I hope to blog soon about other progress on the ES6 and ES7 “Harmony” fronts. A few clues can be found near the end of my Strange Loop slides.

/be

PS: I colorized the code examples in this post using the fun Prism.js library adapted lightly to ES6. Many thanks to Lea Verou, et al., for Prism.js!