15 April 2012

The infernal semicolon

Most of the comments in this semicolons in JS exchange make me sad. The code in question:

  clearMenus()
  !isActive && $parent.toggleClass('open')

relies on Automatic Semicolon Insertion (ASI) and so cannot be minified except by parsing fully (including ASI), observing the significance of the newline after clearMenus(), and inserting a semicolon when stripping that newline.

Some argue that JSMin has a bug. Doug Crockford does not want to change JSMin, and that’s his choice.

FWIW, I agree with Doug’s canonically grumpy tone if not his substance; more below on the substance.

I also agree with @cramforce and @jedschmidt that the && line is an abusage, allowed due to JS’s C heritage by way of Java, but frowned upon by most JS hackers; and that an if statement would be much better style (and, I take it, help JSMin do right). But this particular criticism is too ad hoc to help resolve the general “Let me have my ASI freedom and still minify, dammit!” debate.

Doug goes on to say:

TC39 is considering the use of ! as an infix operator. This code will break in the future. Fix it now. Learn to use semicolons properly. ! is not intended to be a statement separator. ; is.

The !-as-infix-operator idea is proposed as syntactic sugar for promises, which may or may not make it into Harmony with that exact syntax, or with any syntactic sugar at all.

Doug’s right that ! is not a statement terminator or “initiator”. And (my point here), neither is newline.

But search for [nlth] in the proposed promises grammar and you’ll see something surprising about ASI and infix operators: we can add new infix operators in the future, whether new contextual keyword-operators (e.g., is and isnt — BTW these are in doubt) or retasked, existing unary-prefix operators, provided that we insist on [no LineTerminator here] immediately to the left of any such infix operator.

(In ECMA-262, [no LineTerminator here] is used in so-called “restricted productions” to make contextually-significant newlines, e.g., after return without any expression of the return value on the same line.)

This future-friendliness to new infix operators comes directly from ASI as a newline-sensitive error correction procedure, as the example at top demonstrates. Try other examples using a leading identifier on a well-formed second line and you’ll see the same effect. Removing the newline introduces an early error, which creates homesteading space for new infix operators in a later edition of ECMA-262. Examples:

let flag = x is y;  // no n before 'is'!
x ! p = v;          // Q(x).put(’p’, v)

An aside on coding style: if we add new infix operators used in restricted productions, this gives weight to the JS coding style that puts infix operators in multiline expressions at the end of continued lines, rather than at the beginning of continuation lines.

So while I agree with Doug on those two lines of code from Bootstrap (an excellent JS library, BTW) exhibiting poor style, it is not the case that such code as written could break in the future, even if we were to adopt the !-as-infix-operator strawman. The first line terminator in that example is indeed significant.

The moral of this story: ASI is (formally speaking) a syntactic error correction procedure. If you start to code as if it were a universal significant-newline rule, you will get into trouble. A classic example from ECMA-262:

a = b + c
(d + e).print()

Similar hazards arise with [, /, and unary + and -. Remember, if there wasn’t an error, ASI does not apply.

This problem may seem minor, but JS file concatenation ups the ante. For this reason some style guides (Dojo, IIRC) advocate starting your reusable JS file with ;, but people don’t know and it’s easy to forget.

I wish I had made newlines more significant in JS back in those ten days in May, 1995. Then instead of ASI, we would be cursing the need to use infix operators at the ends of continued lines, or perhaps or brute-force parentheses, to force continuation onto a successive line. But that ship sailed almost 17 years ago.

The way systematic newline significance could come to JS is via an evolution of paren-free that makes it to Harmony status. I intend to work on this in the strawman, but not for ES6.

Some of the github issue comments are naive or idealistic to the point of being silly. Since when does any programming language not have syntax arguments? All living, practical languages that I know of, even those with indentation-based block structure and similar restrictions, have degrees of freedom of expression that allow abusage as well as good usage. Language designers can try to reduce degrees of freedom, but not eliminate them completely.

My two cents: be careful not to use ASI as if it gave JS significant newlines. And please don’t abuse && and || where the mighty if statement serves better.

I’ll also say that if it were up to me, in view of JS’s subtle and long history, I’d fix JSMin. But I would still log a grumpy comment or two first!

/be

41 Responses to “The infernal semicolon”

  1. Dave Herman says:

    What you say is true, but we should keep in mind that [No LineTerminator here] restrictions are not free — every time we add one, we add new hazards like the return hazard (where an accidental newline can cause an object literal to parse as a block statement).

    I’m not saying I’m opposed to adding them; I proposed one for modules, after all. But I worry that the restriction will be more costly for infix operators, for example in chained contexts.

  2. Brendan Eich says:

    @Dave: good point, and worth making about adding anything, even if the syntactic space has been cleared by an early error. I agree with you and @jorendorff, and I hope we can put this on the TC39 agenda, that ‘is’ and ‘isnt’ don’t pay for themselves. Object.sameValue or (shudder) .isSameValue should be enough for “egal” use-cases.

    /be

  3. TJ says:

    pleeease please no more sugar :( less is more

  4. Oliver says:

    I still feel that strict mode could have disable ASI — not simply ignoring it (due to the horrible semantic issues) but rather turning any occurrence of ASI into a Syntax Error.

    I’m not saying that’s ideal, I would just prefer a model where ; is either always required or never required and ES has neither. So it goes.

  5. Brendan Eich says:

    @TJ: on infix-! for promises, ‘is’ and ‘isnt’, I agree.

    @Oliver: too late, and we’re not extending strict mode in ES6, rather simply building on it where we can in “1JS” (version-free opt-in via use of new syntax). No more modes!

    /be

  6. Brendan Eich says:

    Note: I wrote “allowed due to JS’s C heritage by way of Java” — but neglected to say that Java does not allow general expression statements. So this was a case of a recessive gene from JS’s grandpa expressing forcefully. Or, when I was told to “make it look like Java”, I made it look like C!

    /be

  7. Oliver says:

    @Brendan: I know, that’s why I said i was sad it didn’t get into strict mode in the first place.

  8. de&&bug( “traces without overhead”); — That’s an abuse turned into a convenience.

  9. Domenic says:

    While `!` for promises (or promises of any sort) seem like a long shot for ES6, I was under the impression `is`/`isnt` were a done deal. They would be nice!!

  10. Brendan Eich says:

    @Oliver: oh, didn’t know you were just beating a dead horse :-P. Good closing Vonnegut catch-phrase for this topic, though ;-).

    /be

  11. Brendan Eich says:

    @JeanHugues: good one, an exception to the rule? A general abusage can have specific good-usage applications (famous writers can pull it off in English).

    @Domenic: is/isnt for harmony:egal are “in” still but at risk. Some believe that they’ll be so underused as to not pay for themselves. The ES5.1 SameValue algorithm (9.2) is not much used within the spec, and of course less so outside it in JS today ;-).

    TC39 and really everyone on es-discuss need to sort out what’s best here. I hope to rely on dherman to make the case against operators.

    /be

  12. Yehuda Katz says:

    The new operators have the unfortunate side-effect of further confusing “equality” in JavaScript. Today, the difference between “==” and “===” is the source of much confusion and bugs.

    We think we solved the problem by saying “never use ==, always ===”, but the message doesn’t make it to every new developer (I suspect it doesn’t make it to most new developers).

    Adding `is` and `isnt` adds yet another kind of equality, with subtly different semantics. We’ll once again think we’ve solved the problem by telling everyone “`is` is the new ===”, but in practice, it will simply increase confusion.

    In general, I am somewhat wary of fixing bugs by adding new syntax alternatives (another example is var vs. let). If the old syntax still works, and existing materials tell people to use the old syntax, there will be a heavy burden on the JavaScript development community to evangelize the differences, which, in many cases, will mean explaining and understanding the differences.

    Imagine the current controversy over semicolons (started by people who fancy themselves spec experts), but applied to something that really matters, like variable scoping.

  13. MumboJumbo says:

    Wholeheartedly agree.

    The usual suspects for code like this come from wannabe “Javascript ninjas” that want to prove themselves (insecurity?) by demonstrating their knowledge of the nuances of the ECMAScript spec.

  14. pav says:

    I’m curious, does ASI actually hurt the performance of JS parsing ? I mean, will there be any performance difference between a code that properly uses semicolons versus one that relies on ASI..

  15. Brendan – for this and many other reasons I came up with a bunch of suggestions for a “more ideal” language, all of which I wrote up here:

    http://news.ycombinator.com/item?id=2044752

    It actually starts with Javascript and compiles to it, because Javascript has become a kind of “lingua franca” and node.js has popularized it. Sure it has a bunch of “WAT” and “wtfjs” moments but overall it’s great for event-driven programming, which is what many of us need these days.

    What do you think of those suggestions? I agree they may take some getting used to, but they would enable IDEs and tools to become way more helpful!

  16. jhuni says:

    “Since when does any programming language not have syntax arguments?”

    Since the 1940s there have been assembly languages with a one-to-one correspondence with machine code and no syntax arguments. The assembly language IPL (information processing language) introduced basic programming features like list processing and higher order functions in 1954. Based upon these foundations John McCarthy created the Lisp programming language in 1958 which was the first high level language without syntax arguments.

  17. Brendan Eich says:

    @Yehuda: agreed, which is why is/isnt are in trouble. More as that develops.

    @Pav: benchmarking ASI vs. no-ASI builds might show some cost but my bet is that it’s in the noise.

    @Gregory: JS remains heavily hand-coded and I don’t expect this to change, so we need to fill in some gaps for human users, as well as fill gaps hindering compilers.

    If you are game, I encourage you to develop your language. @jashkenas (with a bit from me) gave a talk at JSConf.us 2011 on this topic of roll-your-own transpiled-to-JS language:

    http://blip.tv/jsconf/jsconf2011-jeremy-ashkenas-5258082

    @Jhuni: yes, someone on twitter pointed out assembly, I should have excluded it. As for Lisp, alas there are syntax arguments, especially in modern dialects with extensible syntax and full hygienic macros.

    /be

  18. Brendan Eich says:

    It’s clear from twitter wars that my use of “error correction procedure” is being misconstrued as “user error …” (as in “blame the user”), rather than “parsing error …”. Think of the Dragon Book on error recovery, not something judgmental of user intent. This is all about algorithms per ECMA-262, not “ASI tastes great! LESS FILLING!” wars.

    /be

  19. Dannii says:

    Why do you consider the && line an abusage? It’s simple and concise. And if it is an abusage, then aren’t the very common “func = native || fallback” and “arg = arg || default” also abusages?

  20. [...] be used in its place due to the likelihood of bugs creeping in. Backing up this point of view is Brendan Eich, who we should point out, actually invented Javascript. The issue has raised its head on a bug [...]

  21. fpiat says:

    As JS’s user since 1996 I can’t understand why all this time is lost with discussions like this one at this moment.
    I can’t understand where is going this language that I must defend everyday against the lack of interest of the academic. Such intellectual masturbation on such a topic with such infamous coding tricks really does not help to get things done.
    I think Mr. Eich and Mr. Crockford have other things to do than argue about a wrong problem.

  22. Adding ASI support to JSMin is trivial. I can say this 100% confidence, because I have already done it at my personal fork: http://prettydiff.com/lib/fulljsmin.js

    Demo it at http://prettydiff.com/?m=minify

    I completely understand that Crockford wants to stand his ground and has a rational opinion on the subject. Since adding ASI support is so utterly trivial I would say this argument is a complete waste of energy.

  23. [...] finally arrived at the official JavaScript drama with big players like Brendan Eich and Douglas Crockford involved in the discussion. The start of this is part of the bootstrap [...]

  24. Brendan Eich says:

    @Dannii: && and || are fine in expressions that compute used results. The abusage is using them at top-level of an expression statement for control effects better done via if. See

    http://news.ycombinator.com/item?id=3844577

    @Fpiat: there are non-trivial issues to discuss here. And whatever the cause or lack of good reason for fighting, there was a big brawl. Helping avoid an unnecessary us vs. them conflict is worth some effort.

    Good news — @fat and Crock have already made the desired fixes:

    https://github.com/twitter/bootstrap/issues/3057#issuecomment-5140916 (this was already up when I blogged, sorry for missing it)

    https://github.com/douglascrockford/JSMin/commit/5ca277ea452beae1c25db3bc0ef5c81309a3daf4

    /be

  25. Alan B says:

    Just type a semicolon. It’s not the end of the world FFS.

  26. [...] You can read his whole post, which comments on both sides of the argument, here:  http://brendaneich.com/2012/04/the-infernal-semicolon/ [...]

  27. zokier says:

    @Juhni

    Assembly certainly has syntax arguments. See Intel vs AT&T syntax.

  28. Red says:

    Personally, I deeply appreciate ASI (and I’m not a beginner). Somehow it leads to cleaner code, but I think it is rather a matter of taste. Most of the arguments against it, are rather an attempts to find an excuse than a real technical reasons.

    I’m curious about the disputed piece of code. Is this abuse of the syntax too?

    do_something() || die()
    do_it() && then_do_that()

    (It’s a borrow from a unix shell) If you rely on side effects, then this seems to be much better than using “if”?

    There is one issue that is bothering me for a long time. Along with strict mode, has been banned keyword “with ()”, unfortunately, no one suggested any alternatives. Statement “with” combined with “new Function ()”, opens an easy way to create a very powerful tools (eg spreadsheet). Unfortunately, function constructor does not allow you to specify the environment.

    Another thing that I miss the call in style “with (this) {}”.

    Sorry for the small offtopic, but as I mentioned the problem haunting my mind a long time.

  29. Brendan Eich says:

    @Zokier: good point, but *within* one Assembly language you find few degrees of freedom over which to wage style wars. Perhaps only naming and literal notation!

    @Red: ‘if (C) E;’ short-circuits just like C&&E, and ‘if (C); else E’ is C||E. The critique from @cramforce, @jedschmidt, and me is about style, or “usage vs. abusage” (see Eric Partridge on English usage). An abusage is an abuse allowed by the grammar of a language, but not favored on style grounds.

    Also, Unix shell users want shorthands, esp. for one-offs they type interactively. JS, to be used by others, tends to favor readability. More social pressure on JS, in my experience, due not just to github but to view-source and the reach of the Web.

    Hard to justify ‘with’ for spreadsheets — search for functional reactive programming in JS and note no ‘with’ required. Ditto ‘with(this)…’ — use ‘this.foo’ or ‘var t=this;… t.foo’ and the modern JS engines will reward you with v. high perf. Engines punish ‘with’ via v. low perf.

    /be

  30. Red says:

    I understand your point now.

    So, the code is not bad in itself. The problem is rather that it’s “sounds strange” for programmers accustomed to certain conventions (Unfortunately, the same argument applies to ASI).

    For me “if (C) E” and “C&&E” are not equivalent, even if they would end as the same vm code. I do not expect any side effects from “if(C)”, in contrast to “C&&E” which is rather procedural code.

    About “with”. Spreadsheet wasn’t good example. I was thinking more about code generation, perhaps a better example would be page templates.
    Maybe a good alternative to “with” will be ability to set global environment for the function (sandbox)?

    Thank you for pointing out the FRP, just starting to read about this technique.

  31. Brendan Eich says:

    @Red: no side effects in the example from Bootstrap at top of my blog post. Same as if (!isActive) $parent.toggleClass(‘open’) (which is how the tip code reads).

    Templates too can use Function to generate a function whose body refs parameters to the generated function by name, without need for dynamic scope hacks.

    Sandbox support is part of http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders — Brendan-Bob says check it out!

    /be

  32. Mook says:

    Huh, I’m surprised to find that you’re on the pro-semicolon side; mostly because some of your old blog posts (e.g. “Harmony of My Dreams”, from Jan 2011, http://brendaneich.com/2011/01/harmony-of-my-dreams/) skip them. But now that I look, the slides you posted later all have semicolons in them…

    (This isn’t an argument about the actual semicolons – I prefer them – but just an observation that pleasantly surprised me.)

  33. Brent says:

    @Dannii. Exactly.

  34. Brendan Eich says:

    @Mook: am I really on a “pro-semicolon side” in a phony dialectical struggle? Please note the title of this post. See also the positive remarks I’ve made about NPM style, used by those tall enough to ride.

    The idea that semicolons are “optional” is careless hype, or a myth. You clearly need semicolons to separate two expressions intended to be separated by a statement boundary, that would otherwise combine into a larger expression. Yet with intentional and careful use of ASI, you can avoid most semicolons, a la NPM style.

    ASI is part of JS, it is not going away. JSMin should (and now does, at least in one respect) cope. Nevertheless, poor style and human error can combine to make traps for the unwary maintainer (read: any human).

    Therefore I suspect “best style” for a large cohort is “use semicolons as in C”, but I can’t prove this and I do not insist on it. And neither does JS, so any such large cohort will generate omitted semicolons (usually trailing before a closing } or EOF, which are harmless). Not everyone JSLints.

    @Brent: http://news.ycombinator.com/item?id=3844577

    /be

  35. [...] “The infernal semicolon” by Brendan Eich (brendaneich.com) [...]

  36. Kevin C says:

    If ESNext includes pragmas then for dev’s who don’t wish to pay the ASI tax:

    use noasi;
    // mandatory semi-colons for statement termination
    // disallow other quirks (like the ‘hole’)

    Also, a pragma for the security focused:
    use ses;
    // Secure ecmascript
    // creates object ses
    // no global object (or only a frozen global with ES primitives)
    // host resources explicitly requested – like Dart.

    const document = ses.dom.document;

  37. Brendan Eich says:

    @Kevin: pragma syntax was written up at

    http://wiki.ecmascript.org/doku.php?id=harmony:pragmas

    and promoted to Harmony status, but I believe we agreed to defer the ‘use X’ syntax early this year, since no other proposal in Harmony yet uses it. There was some ‘use noasi’ talk last year on es-discuss, but no ‘use ses’ proposal that I can recall.

    In general with 1JS (the current version-free opt-in model where using new syntax is its own opt-in to ES6) we are trying to avoid any more modes akin to ES5′s “use strict” pseudo-pragma. Pragmas for things like turning off ASI make yet another mode.

    /be

  38. Kevin C says:

    I appreciate the desire not bifurcate the language with modes.

    But the suggested pragmas do *not* effect the core runtime/kernel. And I get the feeling this is what vendors are wary about updating too radically.
    - noasi – only touches the parser. Pretty straightforward
    - ses – only effects the initialisation of the global object. Also pretty straightforward.

    Part of the thinking behind these suggestions is to take the good bits from Dart and incorporate them into ESNext. And make Dart redundant.

  39. Brendan Eich says:

    @Kevin: strict mode is code (static) attribute, mostly compiled away. A few runtime semantic shifts, yes. Same with ‘use ses’.

    Implementors are not eager to add pragmas, and few developers are calling for them. True fact!

    /be

  40. [...] in 2009, but it appears that as of 2012, the pissing match is still going strong.  I checked Eich’s blog yesterday and he and Crockford are still going at [...]