As I noted last year, one of the biggest challenges to open source software has been the patent status of video codecs. The most popular codec, H.264, is patent-encumbered and licensed by MPEG LA, under terms that prevent distributing it with open source products including Firefox. Cisco has announced today that they are going to release a gratis, high quality, open source H.264 implementation — along with gratis binary modules compiled from that source and hosted by Cisco for download. This move enables any open source project to incorporate Cisco’s H.264 module without paying MPEG LA license fees.
We are grateful for Cisco’s contribution, and we will add support for Cisco’s OpenH264 binary modules to Firefox soon. These modules will be usable by downstream distributions of Firefox, as well as by any other project. In addition, we will work with Cisco to put the OpenH264 project on a sound footing and to ensure that it is governed well. We have already been collaborating very closely with Cisco on our WebRTC implementation, and we are excited to see Cisco deepening their commitment to the Open Web. Or, as Jonathan Rosenberg, Cisco CTO for Collaboration puts it,
Cisco has a long-standing history of supporting and integrating open standards, open formats and open source technologies as a model for delivering greater flexibility and interoperability to users. We look forward to collaborating with Mozilla to help bring H.264 to the Web and to the Internet.
Here’s a little more detail about how things are going to work: Cisco is going to release, under the BSD license, an H.264 stack, and build it into binary modules compiled for all popular or feasibly supportable platforms, which can be loaded into any application (including Firefox). The binary modules will be available for download from Cisco, and Cisco will pay for the patent license from the MPEG LA. Firefox will automatically download and install the appropriate binary module onto each user’s machine when needed, unless disabled in the user’s preferences.
Interoperability is critical on the Internet, and H.264 is the dominant video codec on the Web. The vast majority of HTML5 streaming video is encoded using H.264, and most softphones and videoconferencing systems use H.264. H.264 chipsets are widely available and can be found in most current smartphones, including many Firefox OS phones. Firefox already supports H.264 for the video element using platform codecs where they are available, but as noted in my last blog post on the topic, not all OSes ship with H.264 included. Provided we can get AAC audio decoders to match, using Cisco’s OpenH264 binary modules allows us to extend support to other platforms and uses of H.264.
While Cisco’s move helps add H.264 support to Firefox on all OSes, we will continue to support VP8, both for the HTML video element and for WebRTC. VP8 and H.264 are both good codecs for WebRTC, and we believe that at this point, users are best served by having both choices.
Of course, this is not a not a complete solution. In a perfect world, codecs, like other basic Internet technologies such as TCP/IP, HTTP, and HTML, would be fully open and free for anyone to modify, recompile, and redistribute without license agreements or fees. Mozilla is fully committed to working towards that better future. To that end, we are developing Daala, a fully open next generation codec. Daala is still under development, but our goal is to leapfrog H.265 and VP9, building a codec that will be both higher-quality and free of encumberances. Mozilla has assembled an engineering dream team to develop Daala, including Jean-Marc Valin, co-inventor of Opus, the new standard for audio encoding; Theora project lead Tim Terriberry; and recently Xiph co-founders Jack Moffitt, author of Icecast; and Monty Montgomery, the author of Ogg Vorbis.
Cullen Jennings, Cisco Fellow, Collaboration Group, says:
Cisco is very excited about the future of royalty free codecs. Daala is one of the most interesting ongoing technical developments in the codec space and we have been contributing to the project.
At Mozilla we always come back to the question of what’s good for the users and in this case that means interoperation of copious H.264 content across OSes and other browsers. We’ve already started looking at how to integrate the Cisco-hosted H.264 binary module, and we hope to have something ready for users in early 2014.
Watch this space for more exciting developments in WebRTC, Daala, and open web video.
/be
cool!
great news 🙂 so this will be pre-installed in firefox in win/mac/linux? Or will it ask us to download it from cisco’s servers when we go on a webpage that uses the tag and uses the h264 codec? Will this change anything for firefox o.s users?
So whilst Firefox support for Windows XP (along with Chrome’s) will extend beyond April 2014 when Microsoft stops supporting XP, to get Firefox supporting h.264 on XP, you still need a AAC solution?
Any chance you would encourage h.264 and Opus as a marriage that works on XP? Presumably that would take a substantial promotional effort in order to get Opus more widely supported in encoding software (which AFAIK hasn’t quite happened thus far?).
This is very encouraging news!
The “Provided we can get AAC audio decoders to match” bit is less encouraging though. Are Cisco (or others) working in that direction?
So, let me get this straight.
Firefox will, by default and without asking, go online and fetch an executable binary from the cisco website and install on my machine?
If my understanding is correct, I will no longer be using firefox on any machines under my control. I really hope this is not what mozilla is planning to do.
Will the codec binary run in a process sandbox for security, or will it link directly into the main Firefox application like old-school plugins?
Great to hear there will be a reliable option for H.264 in Firefox.
In terms of new codec, like Daala, the big issue will be getting hardware support. If you can’t get new iOS and Android devices to support hardware decoding then you’ll have a hard to time getting mass adoption.
https://www.youtube.com/watch?v=K6JshvblIcM
VP9 is superior and blows H.264 out of the water with higher compression and less bandwidth usage, why would you guys use an outdated codec designed in the 2000s when VP9 is better and doesn’t have MPEG-LA cloud over your head?
Out of curiosity, is there any particular reason why Cisco can’t just adopt the x264 project instead, which is a proven OSS implementation? That would surely solve the need to reimplement the main/high profiles? It just seems strange to open the doors to binary plugins if Cisco WANTS h264 in WebRTC, and there is already a top-of-the-line codec implementation available for everyone to use.
Also, what of AAC? Any chats with Cisco to try to solve that one, or are they only interested in making WebRTC H264-compatible?
@Sam: not preinstalled; downloaded for Firefox users whose OS lacks H.264. Firefox OS and Firefox for Android use the OS codec.
@PD, Olly: AAC is the audio codec used for a ton of H.264 HTML video content, Opus not going to displace soon enough. But we’re working on it and I think we’ll solve it. More when I can say more.
@David: you can turn download off with a pref. More to the point, because the BSD-licensed source code is available at https://www.openh264.org/, you and others can verify the compiled bits come from that source, no malware or spyware added. We will organize community auditing of this sanity check, and the binary modules will be cryptographically signed so Firefox can verify their integrity.
@Brion: TBD at this point, but E10S-redux is coming along well so it is an option.
@Joseph: see OTOY’s ORBX for an alternative approach to burning (and freezing for the life of the hardware) a codec in SiO2.
/be
In your last reply, you said that FxOS and Firefox for Android will use the OS’s codec, but FxOS *is* our code. Were you implying the hardware support? I take it then we can rely on all FxOS devices to possess such hardware support?
On a sidenote, can we use our relationships with FxOS partners to potentially encourage including hardware support for Daala (or other free codecs)?
@Caspy7: yes, Firefox OS (already shipping this way) uses h/w codec, fees paid by ODM/OEM. For even a low-end device, taking digital video shorts with the camera is table-stakes, and encoding >> decoding, so the h/w codec is there.
We definitely need to make Daala winning enough that it goes into h/w, and the lower total cost due to no patent fees helps it win on top of its technical merits. Again, we also work on JS/WebGL-based codecs, a la Broadway.js (H.264 in JS via Emscripten) and OTOY’s ORBX.
In the longer run, if FPGA-on-chip or simply fast-enough GPUs & vector units can handle the DSP compute load, and JS can do non-critical-path serial supervision, then downloadable codecs may prevail. (They have other advantages, including notevolving freely as web content does.)
/be
Great news!
Will OpenH264 codec support GPUs?
We’re chatting over at Wikimedia Foundation about if and how this changes the codec landscape for Wikipedia and Wikimedia Commons.
It seems that without audio, both WebRTC-based conferencing-like applications and general A/V playback of existing content won’t actually be possible.
Is similar AAC codec support being worked on, or is there an alternative non-patent-encumbered audio codec that Apple and Microsoft are expected to support?
Or is that still up in the air?
@Brion: stay tuned on AAC for HTML video, but please note that WebRTC uses Opus not AAC, and WebRTC does not have piles of extant/legacy content. Therefore your statement including the word “both” does not make sense to me.
Opus: https://www.opus-codec.org/ — it really is superior.
/be
@VP9: We are working with Google on VP9 too, but it does not blow away H.264 on encoding when h/w implements the codec. Asymmetry (encoder >> decoder in costs) matters, hardware matters. Even now with SoC vendors doing VP8/9, if you read the fine print all I know of are using software (sometimes on a second ARM core) for encoding.
On desktops and laptops, this is less of an issue, if not a non-issue (everything costs, but de-minimus costs can be tolerated in a given equilibrium regime). On mobile devices, especially low-end, it’s an issue.
/be
@Brendan: thanks for the clarification re: Opus support in WebRTC — I hadn’t been aware that Opus was actually going to be implemented by all browser makers. That clears up the WebRTC end, but we’re more interested in general media playback and encoding; until H.264+Opus will be playable in and Opus in on Apple & Microsoft devices we’ll still need to look in other directions…
So, how can I make sure the precompiled binary doesn’t include an NSA backdoor?
@Brion: WebRTC, never mind Opus, is not yet cross-browser. H.264/AAC for HTML video should be, soon. More when I can say more.
/be
@Avih: great question, and it applies to Firefox, Chrome, and other browsers. But in the case of Firefox for Linux at least, and for Cisco’s OpenH264 binary modules, we can audit: get matching revision of the open source, compile with the same (bootstrapped from open source) clang or gcc toolchain, and compare bits.
Profile-guided link optimization and cryptographic signatures complicate things slightly, but nothing is fatal here. Even MSVC can probably be trusted not to inject a trojan targeted only at one or another Firefox or OpenH264 binary.
Topic for a near-term blog post. It’s important that communities around the world audit products built from open source. Don’t (just) trust us, audit us! Open source (including compilers and runtime libs) matters.
/be
So, leaving OSX and Windows aside, what this means is that if you want a legal h264 decoder for your system, you have to hope that Cisco will provide binaries for your system. Want to bet they won’t provide them for *bsd, and non x86/arm Linux?
In all seriousness, I think we should stick with gstreamer on Linux.
Assuming an AAC implementation becomes available for use, does this make Firefox’s GStreamer support almost redundant? The remaining reason to keep GStreamer support would be for MP3 playback. If AAC happens, is there any prospect of the same thing happening for MP3?
Dropping GStreamer support wouldn’t necessarily be a bad thing on Linux, if only because some distributions like Fedora don’t enable GStreamer in their Firefox builds because GStreamer 1.0 support isn’t there yet.
Gstreamer is currently used to support H.264, AAC and MP3. Gstreamer supports both software and hardware decoding through VA-API. There is some work to be done to take advantage of the hardware decoding but I’d like to see hardware decoding on Linux in the future.
Fedora appears to have gstreamer-0.10 support so Fedora users could use the Mozilla build. There is a patch in bug 806917 for gstreamer-1.0 support so I’m hoping it is not too far away.
To avoid Patent Racket I should install and execute binary code controlled by a company I don’t trust? That’s crazy! I use GNU/Linux and I don’t want all that spyware that proprietary code gives you, and the fact that the sources are Free software but I have to blindly install a binary that can have NO relation with that code is madness!
We don’t need their code, having the specs is enough to decode it with free software, what we need is STOP THE SOFTWARE PATENT MADNESS, or at least do an except for Free software.
With this cisco move we will undermine a Free(dom) codec success and enslave us even more in that MPEG-LA-band hell
So is this the end of flash player?
Stupid question, but if the codec is available in source form – even with a permissive license – why can’t it be included directly in Firefox?
Maybe more stupid question, Firefox has code to use the system codecs (GStreamer, DirectX, …), but it is not activated (only whitelisted codecs work). I’ve commented out a few lines, and my Firefox (on linux) can play any format my distribution supports, including H.264. I wonder why that’s not just used.
You call a binary blob open source just because a wrapper/loader is open? This will end up cancelling other projects, as a working solution is available. And you will license every single future codec, because this move is a proof that former open source projects can be forced into a license troll business scheme.
Essentially you’re delivering a trojan horse, maybe a real one for NSA/cisco, maybe one that only attacks real royalty free codecs. And yes, it will reduce trust in firefox and improve some personal career opportunities in exchange.
@ Ray,
I’ve seen some calculations suggesting that MP3 can now be implemented without patents (though there are still some more peripheral patents in play).
https://docs.google.com/spreadsheet/lv?key=0AiObroK0ZHF-dDRzbUtZYk01bEdpZGxWeTlYUlpxS2c
It’s seems like something that Mozilla might want to get behind if true. Maybe even Xiph will welcome it into their stable of free-codecs, which would be ironic but possibly pragmatic since it’s so widespread it’ll often be better choice than AAC if you care about royalty free codecs.
Wtf? Binary files for Firefox for decoding proprietary data formats? That’s not the reason why I’m using firefox.
Will the location(s) searched by Firefox for the module be documented somewhere?
E.g. so that someone could download it once and then copy it to a system wide location or build the module from source and have Firefox find it?
@Jason: It is not documented because it doesn’t do anything special or unusual. Gstreamer is loaded via a call to dlopen which searches the system library path. On Linux this is specified using /etc/ld.so.conf, LD_LIBRARY_PATH, etc. For more details see the man page for dlopen.
Which distro and release are you using and what versions of Gstreamer are available?
@Brendan – thanks for the hard work. Anything that helps HTML 5 video become cross device or browser is great particularly as you seemed to have kept the new addition open (source)
@Markit: you can disable the download, as noted. You can also, and we will set up automation and community-based (independent) effort to do this, audit the binary blobs based on the open source and verified compilers to be sure no malware or back door has been injected.
@Paul: Flash is going down on Adobe’s own say so, first killing it on mobile in November 2011, then with further announcements and changes.
One might credit Steve Jobs and his “thoughts on Flash” with the first death-blow, but whatever the case, this Cisco news is far in time and space from having a causal relation to Flash’s decline. It’s more like a symptom, but not just about Flash — it’s about HTML5 video.
Flash content still matters on desktop, wherefore Shumway (https://areweflashyet.com/).
@Jason: because then Mozilla would have to take a patent license at a fee from MPEG LA, and (more important, since we could afford paying, however much we do rightly object to paying) our downstream open source projects would have to negotiate and buy their own licenses too.
On your Linux coming with H.264, see Anthony Jones’ comments.
@Bill: no, Cisco is really open-sourcing under BSD license the entire codec, and (see answer to Markit above) the binary blobs can be verified based on the source.
BTW, I think we will arrange community auditing of Mozilla’s *Firefox* builds where feasible (on Linux, at least). Why do you trust us not to have been forced to back-door Firefox and gagged from saying we were forced? Well, ok, if that were to happen, I’d quit and my last act would be to say why. I bet people trust others than me at Mozilla to do likewise.
As I wrote above and on HN, open source (fully open, all bits — also the compiler and linker) is important for auditability. I’ll blog on this, Andreas Gal and I have a draft pending.
@Dave: I’ll point the Xiph guys at your comment and invite them to comment.
@Karl: see my answer to Markit above.
@Kevin: yes, the location of the blobs will be public, AFAIK. See also Anthony’s last answer.
/be
Will I be able to make Firefox use a copy of OpenH264 I compiled myself, even if differs from Cisco’s build, without having to rebuild Firefox? I’d like to be able to easily make modifications in case Cisco tries to add anything user-hostile to their implementation.
I am asking from a technical viewpoint, not a legal one. I realize that if I build my own copy, it won’t be covered by Cisco.
If that is possible, maybe I could also make an OpenH264-compatible library that uses x264 and maybe FFmpeg for potentially better quality and/or performance. 🙂
@Eric Hosmer:
There are two technical pieces here:
1. Constructing a binary module with a compatible interface.
2. Getting Firefox to load it.
Constructing the module should be relatively straightforward. Obviously, if you start with the OpenH264 source code it should “just work”. We’ll be documenting the interface so you should be able to make an ffmpeg-based binary module match the interface, but (as usual with compatibility issues) you will probably run into some glitches that need debugging, but there’s no in principle reason it shouldn’t be possible.
The more interesting question is getting Firefox to load the binary module. For obvious reasons, we’ll want the official binary modules to be digitally signed, so there will need to be some mechanism to override the signature check and get Firefox to load your version. We haven’t really thought through the details of that yet (since we haven’t even designed the whole signature mechanism yet), so I don’t have an answer for that, but I don’t see any reason why we couldn’t do something here. E.g., something in about:config.
@Brendan: I see, thanks.
@Eric Rescorla: maybe you could only check the signature if the codec was downloaded by Firefox itself, i.e. to verify it is the thing you are expecting, but load a system wide installed version as-is since the system will already have either verified the signature [1] or it is the user’s own build.
[1] I expect distributions to ship a downloader package that fetches the codec upon installation, like they currently do for Flash.
@Eric Hosmer: If you have the technical skill to build your own codec module, I would think you could just build Firefox itself with any codec you like staticly without bothering with dynamic downloads or module verification. You could even do this today if you mimic the VP8 interface without waiting for the official codec plugin interface.
@Mo Zanaty: Firefox is a large, rapidly changing program. I’d rather not have to modify and recompile Firefox with every update. I’d rather not have to compile OpenH264 either, but I don’t want Cisco to have exclusive control over what source is used to build the binary.
Although you did make me realize something I didn’t think about. Will Firefox use system codecs (i.e. GStreamer) for WebRTC H.264 where it is available, like it does for the HTML5 Video tag? If so, then I am a lot less concerned about this binary blob, since it is not even really needed.
No matter how much they claim the binary and source match, there is absolutely no way to know this for sure, and no real reason to trust them – the binary is effectively non-free.
Software that goes out and downloads non-free software, or has the option to do so, cannot be included in a free GNU/Linux distribution: https://www.gnu.org/distros/free-system-distribution-guidelines.html so goodbye Firefox. Or hello to doing more work to maintain a Firefox fork.
@Roman: We’ll use gstreamer on Linuxes that come with H.264 support as Anthony Jones noted. Downstream distros that reject free-as-in-gratis binary plugins already must make do without Flash, so they lack fallback (object inside video tag) support and without H.264 cannot display HTML video at all.
Firefox can be configured without any binary module download, which should satisfy those free system distribution guidelines. If not, what is the alternative? Don’t say “Chrome”, it is far more unfree. chromium as a base for an alterna-browser still needs to handle HTML video….
/be