Ragavan Srinivasan’s post about the forthcoming Mozilla Marketplace for Open Web Apps inspired me to write about Mozilla’s surging Web and Device API standards work.
A bit of background. Mozilla has always contributed to web standards, going back to the start of the project. We co-founded the WHAT-WG to kick off HTML5. As readers of this blog know, we are a leader in JS standardization. We have some of the top CSS and layout experts in the world.
In the last eight months, our efforts to extend the web standards to include new APIs needed to build compelling apps and OS components on mobile devices have really caught fire. B2G and Open Web Apps are the fuel for this fire.
So I thought I would compile a list of emerging APIs to which we’ve contributed. In citing Mozillans I do not mean to minimize the efforts of standardization colleagues at Google, Microsoft, Nokia, Opera, the W3C and elsewhere. Standards are a multi-vendor effort (although excluding WebGL [see UPDATE below] one shiny name is conspicuously absent from this list).
The Mozilla contributions are worth noting both to acknowledge the individuals involved, and to highlight how Mozilla is championing device APIs for the web without having a native application stack blessed with such APIs on offer. We see the Web as quickly evolving to match native stacks. We have no other agenda than improving the Web to improve its users’ lives, including Web developers’ lives — especially mobile users and developers.
As always, standards in progress are subject to change, yet require prototype implementation and user-testing. Mozilla remains committed to playing fairly by not forging de-facto standards out of prototypes, rather proposing before disposing and in the end tracking whatever is standardized.
Here is the list, starting with some 2011-era work:
- Geolocation, with Google contributing the editor and Firefox (thanks to Jay Sullivan leading the charge) implementing early.
- WebGL (UPDATE: Chris Marrin of Apple edited) and typed arrays.
- Gamepad API. Co-editor: Ted Mielczarek. Mozillans are also contributing to Pointer Lock.
- Screen Orientation. Editor: Mounir Lamouri.
- navigator.getUserMedia. Co-editor: Anant Narayanan
- Battery Status (in Last Call). From the Acknowledgements:
Big thanks to the Mozilla WebAPI team for their invaluable feedback based on prototype implementations.
- Media Capture. Fabrice Desré prototype-implemented in Gecko.
- Network API. Editor: Mounir Lamouri.
- Web Telephony. Ben Turner, Jonas Sicking, Philipp von Weitershausen.
- Web SMS. Mounir Lamouri, Jonas Sicking.
- Vibration. From the Acknowledgements:
The group is deeply indebted to Mounir Lamouri, Jonas Sicking, and the Mozilla WebAPI team in general for providing the WebVibrator prototype as an initial input.
- File API. Editors: Arun Ranganathan, Jonas Sicking.
- IndexedDB. Editors includes Jonas Sicking.
I did not list most of the HTML5 and Web API work aimed at Desktop Firefox, to focus on the new mobile-oriented additions. There’s more to say, including about bundled-permission follies and how to weave permission-granting (with memorization) into interactions, but not here.
One last note. The CSS vendor prefix brouhaha had, among many salutary effects, the benefit of shining light on an important requirement of competitive mobile web development: CSS style properties such as -webkit-animation-*, however you spell them, must have fast and beautiful implementations across devices for developers to find them usable: 60Hz, artifact-free rendering under touch control. This requires such work as off-main-thread compositing and GL layers.
This is a high technical bar, but we are in the process of meeting it in the latest Firefox for Android and B2G builds, thanks to hard work from many people, especially Patrick Walton, Robert O’Callahan, Chris Jones, and Andreas Gal. Onward!
/be
Please provide that list of Desktop oriented Web API and HTML5 work. Not everyone believes that the mobile web is a tsunami that will take over desktops. It would be interesting to see how much feature development work Mozilla is doing that has no use on the Desktop. Does Mozilla intend to let Desktop Firefox become a second-class citizen? Already it is obvious that vibration, screen orientation, battery, telephony and SMS have limited, if any, use on the Desktop.
Respectfully,
pd
@PD: you’re kidding, I hope — or just new here. Start at
https://wiki.mozilla.org/Standards
We are doing a ton of evolutionary work that is good for desktop Firefox, if not good for *both* desktop and mobile. HTML, CSS, JS, WebRTC, Web APIs such as IndexedDB and WebGL, the list goes on.
What we had foresworn, or had simply neglected, until last summer, were those APIs that are essential to competing on mobile devices — definitely including de-facto standards such as certain of the -webkit- CSS properties (implemented well). That had to change. The change is good news.
Desktop is not going away and we’re investing in it as ever.
I encourage you to think creatively and expansively. E.g., WebTelephony on desktop => Skype integration. Unity of interface is still a guiding principle of the browser, whether desktop browser, mobile browser, or mobile-OS full-screen environment.
/be
@BE: My concern is that there are features the seem to have slipped under the radar. For example native form widgets such as date pickers, colour pickers and so on. Secondly there’s rich text editing. hasn’t evolved since it was created AFAIK. It’s been suggested these features are provided by JS so there’s no need to make them native. Why is that philosophy applied to these features whilst others get full native support?
The most obvious missing shiny name in your post is Apple, despite an Apple engineer being the editor of the webgl spec, and animation+transition specs you’re talking about originating there as well. Trying to say apple doesn’t contribute to modern specs seems somewhat disingenuous.
@PD: HTML5 forms include date and color pickers, and the work is under way (https://wiki.mozilla.org/User:Mounir.lamouri/HTML5_Forms, https://bugzilla.mozilla.org/show_bug.cgi?id=446510 [date], https://bugzilla.mozilla.org/show_bug.cgi?id=547004 [color]).
I do not believe these have been de-prioritized due to mobile vs. desktop. Indeed these input elements benefit both desktop and mobile. B2G wants date picker, for example.
Last I heard we were waiting for rich text editing spec from Aryeh, checking… looks like @dbaron is ok with spec, we need implementation. Gecko’s editor code needs an overhaul. @ehsanakhgari was on that case for a while.
Thanks for raising this — again I do not believe it has been neglected because of mobile, which also wants (not just for tablets, but at least there) rich text editing. I will stir the pot.
/be
@Oliver: I dodged the CSS bullet by focusing on mobile Web APIs. Ignoring WebGL (which I did cite), I see a lack of Apple contribution among the recent mobile-oriented Web API work.
I stand by this observation, and indeed it’s an open secret in Silicon Valley that “[Apple] management says keep out of the web stack” (at least in the past; I heard this was part of a comment in a “rdar” internal Apple bug report about a device API privileged to Obj-C; this from a reliable source no longer working inside the firewall).
On the WebGL item, I didn’t list Chris Marrin as editor, you’re right — but I also intentionally and with explicit disclaimer didn’t list Google editors either. Yet I did list WebGL. Sorry, this was an oversight, not disingenuous. I will give Chris/Apple a shout out in an update.
Apple going back to the WHATWG founding (well, a few months later, in public :-|) has been a source of great innovations including <canvas>, many of which were promptly proposed as standards.
Good though that all has been, and I’m grateful for it — including the pre-standards-track innovations — I have to say this: Even better would be proposing before shipping, providing specs promptly, retiring vendor prefixes, and not asserting patents. But I’m told none of that is permitted.
And from recent w3.org mailing list traffic it’s clear that Apple does not have enough standards people on staff. See
https://lists.w3.org/Archives/Public/www-style/2012Feb/0356.html
I know Dean from his w3.org days and he’s a good guy. His mail makes some fair points in defense of Apple, but this sentence is pretty damning: “Despite having billions in the bank, we don’t have the luxury of full-time W3C editors like Hixie and Tab [Googlers]”.
/be
Brenden, Not to toot my own horn, but just want to point out the work to get multitouch up and running using Apple’s no-so-great API’s. The implementation was my one small part, but mbrubeck, smaug, and others have put a ton of time and effort into digging through the horrible documentation and building tests. They put together a document that is compatible with the iPhone implementation (i.e. basically the same reverse engineering we’ve had to do with IE all these years), and extends it with new interesting abilities like radiusX/Y, pressure, and orientation where available. And they’re now enjoying a fine patent battle with the company that released it onto the web to begin with. Its been no small feat on their behalf, and I was kinda sad to see them not get credit here.
@WesJ: no fear, I will credit you, @mbrubeck, @smaug et al. I have more posts coming.
/be
Hi,
First of all great work on the API standardizasion.
I’ve seen some video showing boot to gecko prototypes, but I noticed something, there is no mail app !
Basicly, for me fetching my mail is a key element for a smartphone. As a web-developer I know you only have 1 way to do that :
– using a webmail, but most of the mail providers do not provide any mobile webmail, just because there is no open source mobile webmail that exist (you can only find paid theme for roundcube for instance). This solution is not good enough if you have several email accounts (switching from a webmail to another … and the webmail provider is not going to let you had external email accounts just because he doesn’t want to carry the traffic (and pay for it).
An API could be a solution to solve that problem :
– developing and API allowing a webapp to obtain the right to open “socket” (not websocket) normal socket like what you can do using a telnet client. This would allow an email client webapp to act without relaying on a server side ( HTTP imap gateway). So that people would be able to get a fast mail webapp.
It would be like a telnet API 🙂
Have you planned to make such a thing available ?
Benjamin BERNARD, maison du libre mdl29.net
@Benvii: yes, B2G needs a Mail app and yes, it will be JS/HTML/CSS/img based — so will need new APIs for SMTP/POP/IMAP, including the SSL option. I’m keen to help make the B2G Mail app great. More on this as it develops.
/be
Ideally a developer will be able to develop one set of code and then deploy it as:
a) a web 2.0 web site
b) an app – which can work offline with cached data if needs be.
Users shouldn’t feel there are two separate worlds – web and apps. Rather a spectrum with apps being full screen – with maybe a richer more immersive experience.
If so ECMAScript needs to become better. Or a ‘new’ language – developed in the OPEN with the input of the major players. Not in the dark as with Dart. I think there is a case for this.
@Brendan Eich is there a way to follow that, a mailing list ?
While I realize XULRunner no longer receives much focus in the Mozilla organization, is there any similar marketplace for XULRunner apps?
I provide services to individuals with disabilities. Many of my clients interact with their computers using nothing more than a simple switch – no keyboard, no mouse, no touchscreen. Switch interfaces take on variety of guises depending on what kind of platform the user is interacting with. A switch activation could trigger a mouse click, joystick button, or keyboard event. Just how any given program responds to these events varies widely between programs as well. Take browsers for instance. You can navigate between click-able links and objects using space bar in one browser, and tab in another. Consistent behavior is the exception and causes not only trouble for switch interface users but for keyboard only users (low vision users) as well.
I propose a standard. A user interface API that would normalize interface interactions so that the browser, or the web page, or the web app would conform to the users preferred method of access, rather than imposing its own arbitrary definitions, or confining users to mouse or touch only interactions. It could work like CSS where the users preferences would be read from a simple text file and the app would configure itself to meet the users needs. A web standard that attempts to take this task on is a good place to start. Making the user experience consistent across browsers and platforms seems to be a guiding principle behind web standards, and I think it should be expanded to include all kinds of users who may need more uncommon means for interacting with content. It seems clear to me that desktop only, or platform specific applications are a inefficient way to develop applications that one hopes will reach the greatest audience. Well crafted web applications can be the ultimate cross-platform, browser-agnostic means for delivering to the widest audience. Improving and following accessibility standards could make it even more inclusive.
@Kevin C: you talk about offline vs. “Web 2.0” apps, web vs. apps, but then end oddly by calling for a new programming language. That’s crazy talk! The problem is not JS, it’s the platform APIs for virtualizing local vs. remote storage, and otherwise for operating offline (which cannot be made transparent in all cases, of course). We are working on that.
@Benvii: https://groups.google.com/group/mozilla.dev.b2g/topics?pli=1 is a fine place to follow.
/be
Totally agree with – platform APIs for virtualizing local vs. remote storage – work.
But also agree with the analysis in Mark Miller’s famous email. However the new language should have been designed in the OPEN by the ecmascript JS committee.
Two tracks at ecma would be cool:
1. Improve current JS (which is happening)
2. A trial ‘new’ language. If you build they might come – and if not there’s always 1.
1. or 2. should blur the web 2.0 VS app distinction.
Currently JS VS Objective C/Java-Dalvik.
Preference for 2. But the strange thing is there’s more creative thinking with 1. on the ecmascript committee then there is with 2. Dart. And Dart had a greener field – with the constraint of compiling to existing JS. Though Dart made some good pragmatic decisions.
Off topic:
Anyways – how about repurposing the for loop to work with blocks – whatever the final block syntax (if blocks go ahead).
let lst = [3,4,6];
for (lst) {|k,v|
console.log(“Key is : ” + k + ” Value: ” + v);
}
With a ‘ new’ language you could have blocks …