Mark Hammond’s work to support Python in XUL is nearly done. The DOM_AGNOSTIC2_BRANCH should land in the next few weeks. Already I see many on the PyXPCOM list testing Mark’s fine work, chomping at the bit to use Python in XULRunner.
This brings to mind a hot topic in my recent hacking: infusing JS with Pythonic generators and iterators, including array comprehensions. Brief taste:
js> function count(n) {
for (var i = 0; i < n; i++)
yield i;
}
js> g = count(10)
[object Generator]
js> g.next()
0js> g.next()
1
js> two_to_nine = [i for i in g]
2,3,4,5,6,7,8,9
js> squares_to_20 = [i * i for i in count(20)]
0,1,4,9,16,25,36,49,64,81,100,121,144,169,196,225,256,289,324,361
Why borrow from Python for iteration, generators, and comprehensions? I typed this into a wiki recently:
Given the years of development in Python and similarities to ECMAScript in application domains and programmer communities, we would rather follow than lead. By standing on Python
You should consider working with KDE, too (who originally developed KJS, notably).
Brendan, excellent stuff. There is a definite need for a general purpose scripting language which is both easy for novices to pick up and good for powerusers. This scripting language should be for use on the client (browser,desktop,applications) and the server (web, db).
JS2/ECMAScript 4 could definitely fill this need. As you rightly noted JS seems to have some wind in its sails with ActionScript, MAC OS X Dashboard widgets, KDE 4 and QT 4.1 and of course firefox.
It seems like scripting languages are converging on fundamental semantics with a pragmatic
mix of imperative, OO, functional paradigms and support for data typing, xml/sql data handling and programming in the large (packages/modules). Borrowing the core good ideas from python, perl, ruby, scheme is great.
It stuck me when reading the ECMA spec pseudocode that an alternative english like syntax for ECMAScript – but with exactly the same semantics – would be useful. Especially for new users. Both syntaxes could compile down to exactly the same ast tree and convert from one to the other without any loss. For example:
// C curly braces syntax
var x as int = 10;
var y as int = 11;
if (x==10 && y==11) {
doSomething(x);
}
// English like syntax – quite like Lua
var x as int = 10
var y as int = 11
if x eq 10 and y eq 11 then
doSomething(x)
end if
// or even pythonesque indentation
if x = 10:
doSomething(x)
The past 10 years has shown the importance of getting the simple basics correct – which gives a foundation for developers to build rich solutions. MS are trying to do this with the total retooling of their languages and frameworks in vista. ie dotnet clr + xml. The non MS area needs to a have a coherent technology stack. I dont think Java has much of a role to play. Rather:
C -> C++ (or D) -> JS2 + XML/JSON.
It would be tremendously helpful if JS2 could standardize plugin APIs in both C and C++ (and do the latter in a template-smart way). Having simple C APIs accelerated the growth of Python and Ruby tremendously, and (I believe) is crucial to getting JS accepted as a “traditional” scripting and glue language. Standardizing “one way to do it” for C++ (esp. templated C++) would avoid a lot of the grief P&R have gone through.
Another Python idiom that JS lacks, painfully so for me, is sprintf variable interpolation into multiline (triple-quoted) strings. I wind up generating JS using both Python and JS, it’s really easy one way and really hard the other.
Sho: it looks like George Staikos is working to reunify KDE KJS with the WebKit JavaScriptCore fork, so I’m assuming two birds with one stone can be killed there.
kevin: I think the syntax ship has sailed. It would just add parser complexity and potential for confusion to offer alternate syntaxes, although if we end up with the macro system I hope for, users could pick a syntax library and suit themselves. As someone who grew up in the old days of Unix, I can tell you that alternate syntax usually just divides and alienates. Steve Bourne used “C-gol”, a set of C pre-processor macros enabling
IF condition THEN consequent ELSE alternative FI
and festooned the original Bourne shell with it. It was fun at first, but just a barrier to entry for new maintainers and extenders over time.
Pythonic significant whitespace is a bit of a mismatch with HTML. Difficulties with Python in inline script tag content seem likely, especially over time. I see a lot of free-format JS in inline scripts on the web, where indentation is random or missing (all on one line JS is common).
JS is the lingua franca of the browser, and in my opinion it is better to have one syntax. It’s hard enough disambiguating a C-like language that has grown regular expression and XML literals, and which started out with automatic semicolon insertion.
Greg: for Mozilla, XPIDL and XPCOM (https://www.mozilla.org/scriptable/) are the way to interface to “plugins” (generally construed) implemented in other languages, not just C++ but also Python (a C XPCOM binding could be done, and one was started, but there hasn’t been much interest). A truly painless C or C++ binding should work as with OCaml — that is,
ocamlc foo.c
This is a fine goal, but not something for ECMA TG1 to standardize just yet. I would welcome a prototype for SpiderMonkey, if volunteers capable of pulling it off are motivated.
Plugins loaded from the net are a different animal, requiring lots of trust as well as highly compatible, stable APIs on both plugin and browser sides of the fence.
Jon: ECMA TG1 has a “string formatting” item on our wiki. Foundational concerns such as the type system and runtime semantics related to it pull our focus, but we know that standard library components that ought to have been there ten years ago can be even more important to users than fine new things such as categorical sum types ;-). Date is another example, not of missing so much as botched standard library functionality (thanks to java.util.Date, argh).
I’m committed to filling the formatting gap well. There is some tension between Java-like and C#/.NET-like formatting. Other influences such as Python and Perl seem like darker horses to me. Any thoughts on which precedent to follow?
/be
Brendan – thanks for the response. Yes, realistically one syntax is the way forward.
The language Lua does share quite a lot of semantics with JS. Its approach to iterators, generators and continuations is interesting.
https://www.lua.org/pil/
Re your previous post on the type system. The XML integraton with E4X is very useful. The basic built-in data types defined in “XML Schema Part 2: Datatypes Second Edition” are imo quite well done. (Regardless of feelings about XMLSchema overall). Especially the datetime types. For JS2 any chance of synchronising the js and xml schema basic types.
https://www.w3.org/TR/xmlschema-2/#built-in-datatypes
This would avoid the problems that the .net clr and java have in mapping to xml(and sql) types. Could the Date object become a basic type with value rather than object ref semantics. And to define value types as well as reference types. Basically on assignment a copy is made rather than pointing to the same object.
I’d love to see a Python-like concept of “module”, where a script gets its own global scope and that scope is itself a module. This is in contrast to current module systems, which are generally specifically declared (i.e., assigned to an object) and can leak fairly easily and are syntactically awkward, and the namespaces are self-defined, instead of being defined by the integrator. So I imagine:
OtherLibrary = require(here + ‘/other_library.js’);
Where ‘require’ — as in several other languages, like PHP — will return an existing object if it’s already been required (from the same URL), or else load it and return it.
‘here’ refers to the containing directory of the current script; something like that should be available, but it’s not really enough to make this work properly. It works fine when you have a parent-child relationship, i.e., you are including some sub-library that is located relative to the “containing” script. But if you have some “basic” library that several other libraries depend upon but none really owns, then where to find that? You can’t just go looking speculatively at a bunch of locations on the net, like you can on disk (where programming languages typically use search paths).
I almost imagine some kind of super-global script-finding function, that people write (or specialize) based on their deployment. You’d still use the explicit require(uri) when you were doing (effectively) relative imports, but for getting other major libraries you’d hook into this, where a default implementation might look in some single location (and that location might be customarily defined in an inline script in the HTML page, or whatever the containing environment is). So maybe import(library_name, [version]) would call that finder, then require(); and could also be used for a standard library.
Well, the result isn’t that elegent, but I don’t know what is elegent; it’s a hard problem. Python imports can be annoying too. But Python modules are excellent.
Iterators and string formatting would also be excellent additions, as already noted. Then something like __getattr__. Then just little things, like expressions in function names, and maybe a special syntax to make Javascript’s class-like constructs less of a recipe and more of a clear feature — I don’t think that has to mean a new object model, just some more explicit syntax (with implied error-checking) would help tremendously.
It would be nice if, transitionally, there was a translator to old Javascript syntax, with recognizable scripts (i.e., debuggable even in its translated form). I think translating to Javascript is a dumb technique in the general sense, but if it is a temporary state of affairs then it’d be okay.
Brendan, you mention a ” a coherent technology stack….C -> C++ (or D) -> JS2 + XML/JSON.”
Did you ever consider Parrot? Just curious. Please don’t let what’s happened to Perl6 happen to JS2 though.
I tried out the patch on the CVS head, but I’m missing jsproto.tbl and cannot compile. I noticed somebody else encountered this problem several days ago (https://bugzilla.mozilla.org/show_bug.cgi?id=326466#c4).
Any chance this file could be made available?
sorry to ask useless question.. is JS2, Javascript 2, right? If so is there any website explains what it will have? thanks!
Matthias: fixed, see https://bugzilla.mozilla.org/attachment.cgi?id=213828.
Max: not yet, the docs at https://www.mozilla.org/js/language/js20/ are out of date. I’ll blog more shortly.
/be
Thanks for taking the time to post that attachment. I’ve been experimenting with it and have a question. Sorry if you’ve answered this somewhere and I’ve overlooked it.
I’m curious why list comprehensions return array values instead of indices. This seems inconsistent, especially since the “for each” construct already provides this. I do recognize the pragmatic advantage: it’s more concise and is what people typically want. But why not maintain consistency and allow transformations such as this (using the example you gave):
two_to_nine = [i for each i in g]
It also occurred to me that the “for each” syntax could be replaced with a “values” generator, such as:
function values(o) {
for (var prop in o)
yield o[prop];
}
The above example would become:
[x*2 for x in values([1,2,3])]
Thanks.
I like the idea of JS being more pythonic but I really hope it doesn’t borrow python’s tendency not to support nice general solutions, e.g., just supporting yield in iterators (might be wrong term I’ve switched to ruby for this reason lately) and not coroutines in general.
I don’t mean to refight python language battles and I don’t think one has to believe python made the wrong choices to believe JS should take a more fully general approach. I believe there were some technical factors discourage some of these more general solutions in python, e.g., only stackless python could handle these things easily, and when designing these new features in JS this need not apply.
More importantly though JS (of course) servers a very different purpose from python. While python can aim to be a great language for a certain community and tell people who want to program in a lisp/ruby/whatever fashion to go use that language javascript is virtually forced on anyone wanting to do web stuff. Thus JS much more than python needs to satisfy everyone.
If I had to do things over again I would make JS more like ruby but given the current language (not fully OO) python makes more sense as a model.
On another subject I think JS needs a much more robust method to load code/libraries than other programming languages given its status on the web. In particular it would be of great use if JS could support cryptographicly signed library caching. Thus my browser could load rico (or your fav library) at site A in some format that indicates it is a signed library availible to other sites. Now when site B needs rico it has a rico include statement giving some cannonical name, version and cryptographic signature. If this tuple matches the version already cached then it just uses that otherwise downloads from the local source. Such a system could end up being utilized by browser makers to allow for libraries given greater privleges as well.
Matthias: you wrote “I’m curious why list comprehensions return array values instead of indices”, which applies to Python, not to JS with the patch I’m working on. Python defines iteration by a protocol that different data types can map to keys, values, or even (key, value) pairs.
JS’s objects, even arrays, are more like Python’s dicts, so for-in, and any iteration protocol that underlies it in Edition 4, must map over keys, not values. You are quite right that ‘for each (v in o)’, from E4X, is the obvious “fix”, but it’s neither Pythonic nor obviously a value rather than key iteration syntax.
Since Edition 4 is aiming for backward compatibility, we are not redefining iteration over Array to differ (to iterate values). But we are in fact thinking of better ways to expose value iteration than for-each. More in a new blog post.
logicnazi: Sorry, trying to “satisfy everyone” is a recipe for satisfying no one. ECMA TG1 represents several interests and points of view, though, and while we should avoid design-by-committee, we will not do “just classes” or “just iterators” in the new edition. We, or at least I, aim to provide a coherent set of extensions that solve real problems in the existing language, glued together with a sound (I hope) type system.
That does not mean we will support call/cc, or any general coroutine architecture that requires saving multiple activations or whole stacks. There frankly is not real-world call for such generality. What’s more, some TG1 members would have a hard time going “stackless”, including some of the browser-based implementations.
So JS is really not that different from Python in this regard: it needs to support metaprogramming alongside programming-in-the-large, without imposing either onerous optimization costs on implementors, or complexity costs on users.
This is possible, and Python is one informal proof that it can succeed. Although Python is far from simple as it has evolved. JS2 should aim to be a smaller language even (or especially) as it learns from Python.
On the library issue: you are right on. Signing is overkill (a crypto-hash would do), but the main thing is to share standard libraries. However, any web app today must provision its own copies, under its own URIs, of the libraries it uses. This entails edge-caching for high availability and low latency. It’s hard to bootstrap a new system of library-sharing that breaks URI semantics, at least not without guaranteeing high availability.
So my thinking, again subject for a future blog post, is more around supporting https://openjsan.org/ in a massive-infrastructure way (Robert Sayre has proposed this under the jsdmz.org name, but I wonder why we wouldn’t use openjsan). More soon.
/be
I’m the developer of PIES which add import/package mechanism to classic js, currently it mimic the import keyword of JS2 (Waldemar’s draft), from semantic to syntax. And now I’m working on namespace keyword. So I am very curious about what will be changed in the future development. And I also very curious about other new features. Hope the wiki of ECMA TG1 will be public soon 🙂
Nice stuff, but I doubt that JS need it.
Simpler solution will be just add forEach method to Object, almost the same that was already done for Array.
The only change that required is give this operator one more operand – seed. Iteration should be resumed from this seed.
Seed could be item key for array – forEach already have it as third argument.
So the user could simply break iteration and return from function and later resume this operation when needed.
Break can be performed with throw/catch
and resume with arr.forEach(funct, other, seed);
More on this approach in https://okmij.org/ftp/Scheme/enumerators-callcc.html
Another good solution is full-blown continuations, like Rhino already has.
This will be the best solutions, and doesn’t require any new keywords to language.
With continuations it is simple to implement yield as function.
And about lambda syntax.
That could be made very simple to, but not Pythonic.
var double = (a){a+a}
Here – ( – means lambda and looks like lambda, and it is lambda.
And no need to use return keyword in this case,just return last value
[1,2,3].map((a){a+a}).filter((it){it>3})
Simple change to syntax, and everything is ok,and looks nice and what is more important – it looks like JavaScript, not Python or Ruby or something else.
Ok, it’s a little like Haskell, but only a little.
The syntax is plain JS, just parse for after ( and you will get lambda.
I apologize if this is a naive thought.
It seems that it would be easier (better?) to embed Lua in the browser. Do whatever sandboxing, etc. to make it appropriate for the web environment. And if desired create a domain specific language encompassing current JavaScript and if something different than Lua, then also JS2.
I wouldn’t think it would be much more challenging, if at all, for JS users to switch to Lua in the browser instead of JS2. Lua already exists. Its tested, proven and used widely by game engines. It has decent performance. Lua’s syntax and programming model shouldn’t be too foreign to JS developers.
Lua is open source, small, very embeddable. Any and every browser should be able to embed Lua without much difficulty. And I would think easier than creating JS2 from scratch. It seems that it would provide a nice upgrade path for web browsers and enable browser developers to somewhat get out of scripting language/spec/engine development.
Just a thought.
Lua is a fine language and embeddable implementation, but sorry: it is naive to think people will start writing Lua for web pages and apps if we embed it in Firefox. Every browser would have to embed it, more or less at once, after solving the non-trivial security issues in the same way. Then users would have to learn it. Then they would have to stop using JS.
And this doesn’t begin to address the problems with programming in the large that would bite Lua as well as JS1, which are motivating much of the JS2/ES4 work.
As I’ve said in recent talks, the only programming language likely to be found by default in browsers for the foreseeable future is JS. This is a “path dependence” situation, with a vengeance. It’s not that JS is “best”, only that it was “first” and “good enough” — and the costs of replacing it are too high compared to benefits of keeping and evolving it compatibly, and especially compared to opportunity costs.
/be
Regarding the programming in the large issue, i wrote a library [1], which tries to ease this task in JavaScript. It is based on a interception mechanism, which does – next to many other good things – enable type-checking of function call parameters at runtime. The user of a function gets an early feedback of what is going wrong.
Together with the schema concept of my library, it should also be possible to add an Interface concept, if one would want to. One might even want to go so far to test the semantics of a function in a schema to see if it can do the job.
Other than that, i think that this little hack of yours turned out very well, and does not lack too many things! If you look at Java and other technologies, it seems to me, that things are added because people *need* to do something. Usually it is with good intent, but many times it just makes things more complicated with little gain. So, as far as JavaScript 2 goes, pls keep it simple!
[1] https://www.cerny-online.com/cerny.js/