[UPDATE: see Jim’s fair comment below. /be]
I’m pleased to report that OTOY today has announced good news about ORBX.js and the Amazon Web Services ORBX and OctaneCloud AMIs (Amazon Machine Instances, pronounced “AHmees” — who knew?), based on terrific adoption and developer interest:
- Free ORBX and OctaneCloud AMIs forever, not just for a trial period. OTOY will focus higher up the value chain.
- ORBX.js to be open-sourced on github as soon as OTOY delivers on prior promises, I hope by next summer.
- Two major studios have been evaluating ORBX for a watermarked, DRM-free Video-on-Demand service.
- OTOY has an ORBX encoder (built using their own OpenCL compiler) that runs as a small native loopback server, so it can be addressed by browser apps using WebSockets. This is a clever interim solution that avoids plugins and anticipates “ensafened” WebCL, or Rust on the GPU, or a better solution for writing a downloadable and memory-safe encoder — something Mozilla Research has on its agenda.
The deeper meaning here, in my view: a great rift emerged between CPU and GPU in the ’90s, where serial old x86 instruction set compatibility seemed to matter (remember shrink-wrap software?). The need for speed with binary compatibility begot big, power-hungry, superscalar CPUs, while from the SGI diaspora, the GPU went massively parallel.
One consequence of the rift: the rise of ARM on mobile, where binary compatibility did not and does not matter, but power efficiency does.
This rift may yet be healed, and in a way that avoids too much custom hardware (or else we will have to rely on FPGA-on-a-chip).
With enough homogeneity and parallel processing power, always-evolving video codecs, 3D model asset streams, and undreamed-of combinations should be feasible to implement in downloadable, power-efficient, safe code. Perhaps we can even one day kill off some of the video codec patent monsters that are currently burned into silicon.
More to come in the new year; this is just another happy rolling thunder update.
/be
Exciting!
Thanks for the part you play in this and for keeping us informed.
I am curious to know if there are any projects yet to develop ORBX-style encoders/decoders for VP8/9/Opus/etc.
I love how much this has the potential to transform the landscape (in a good way).
“A comparison of ORBX and H.264 quality from an OTOY whitepaper demonstrating that ORBX can achieve less than half the bitrate of H.264 for comparable quality.”
I read the introductory whitepaper ( https://aws.otoy.com/docs/ORBX2_Whitepaper.pdf ) and the claim of ORBX achieving half the bitrate of H.264 is being made out of context. Those images demonstrate that ORBX can achieve slightly better quality at a similar bitrate (what bitrate exactly is not mentioned), not comparable quality at half the bitrate.
OTOY is only claiming roughly half the bitrate with high bandwidth streams (e.g. a 20 Mbps H.264 is roughly equivalent to a 13 Mbps ORBX2 stream on PSNR-HVS-Y with a 1080p24 clip). At bitrates of 1 Mbps to 5 Mbps, H.264 and ORBX2 are quite close with ORBX2 achieving a slightly better bitrate (85% of H.264).
ORBX sounds interesting but I suppose we’ll just have to wait and see what it’s like in the middle of the year. As far as I know there are no public demos available of ORBX.js, which seems a bit odd for something claiming the ability to neatly exploit the open Web platform as a selling point. I had a look on Amazon a while back, but at the time there didn’t seem to be any way to try ORBX out without creating an account and paying money.
Are there any comparisons of ORBX to VP9 and H.265 available? It could be that H.265 is about half the bitrate of ORBX across most bitrates. Maybe VP9 as well, although some claim that VP9 is out performed by H.264 ( https://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf ).
@Jim, Jules of OTOY wrote back when I forwarded your comment, as follows:
Jules wrote: “I am not sure what point Jim is trying to make, other than yes, there is variance – ranging from 40% the bitrate of H.264 HP to 85% the bitrate for the same quality, which is a significant win in both cases (although not the same bitdepth – ORBX2 has 12-bits .vs 8 for H.264). And this is in line with our overall goal we stated in May of being about 75% the bitrate of x264 high profile. If we are replacing a bluray quality stream, the numbers are going to be closer to the 40-50% range, which is significant for the big studios.
The other thing to note is that ORBX2 is designed to decode entirely in JavaScript, and even than at full speed/battery efficiency with WebGL2, which none of these other codecs are targetting (much less encode 100% on the GPU as we with OpenCL) I think ORBX2 beating H.264 HP in every case, even if it is not as much as HEVC might do, is more than enough to satisfy the goal we have of displacing H.264 hold on web and home video.
Given that ORBX is an living codec, and there is already an ORBX3 in the world next year which will offer even better compression than ORBX2 (and full fp32 encoding), I am not concerned about where things stand vs. other emerging codecs – especially as HEVC will probably not be fundamentally improved upon for a decade or more, just as H.264 got stale from 2003-2013.”
Jules again: “I believe it’s possible to get a free introductory usage tier for EC2 which will support at least 2 of our AMIs on their CPU instances . That being said, it only costs 75c to try out a GPU AWS instance. If someone wants to work with a free version of the encoder on his their machine, they should wait for us to release the localhost encoder, which we announced a few days ago.
Also, Autodesk Remote (available for download from ADSK) enables ORBX encoding/streaming from a local machine.”
Jules again: “We only focused on H.264 for the whitepaper. I am not entirely convinced about the quality of VP9 either. I think HEVC may beat us all, but require ridiculously high compute costs, adding the encumbrance of custom encoding/decoding hardware for acceptable performance.”
Brendan closing: comparing video compression schemes is notoriously hard. I’m just jazzed that people can try ORBX.js via AWS today, and that ORBX2 will be open-sourced — then I expect we will have a number of skilled people and groups evaluating it against other schemes — and some of them will patch it to improve it. That’s how codec development should proceed, not via costly custom ASIC “all-in” bets.
/be
Jules wrote: “I am not sure what point Jim is trying to make”
I’m just making the point that the Big Buck Bunny comparison image from the whitepaper is being misrepresented in the CNet article (and in this blog post :-). The image caption implies the comparison is something it isn’t (i.e. not what the whitepaper says it is). It’s lazy journalism by CNet. Doubly so because they don’t link to the same OTOY whitepaper I linked to in their article, which is where I presume they got the image from.
The OTOY whitepaper says the image quality comparison “compares the image quality of ORBX2 with that of H.264. Both images were encoded at a low quality, and to the same size in order to illuminate the differences.” To me that means both the H.264 and ORBX encodings were done at roughly the same bitrates, as opposed to CNet’s implication that the ORBX encode is half the bitrate of H.264 in the comparison image.
I think I see what happened to the CNet article. They probably read the whitepaper quickly and didn’t necessarily have much time to think about it. The paragraph immediately above the image quality comparison section talks about ORBX achieving half the bitrate of H.264 for high bitrate streams and so CNet made the assumption that the comparison image was ORBX at half the bitrate of H.264.
Alternatively, I’ve got it wrong and the comparison image actually is ORBX at half the bitrate of H.264. Either way, putting more detail in the whitepaper will clear it up. 🙂
Jules wrote: “I believe it’s possible to get a free introductory usage tier for EC2 which will support at least 2 of our AMIs on their CPU instances .”
I just want to see ORBX.js working without having to create any accounts or pay any money. As you say, the plan is to open source it in the middle of next year so I’ll have a look then.
Jules wrote: “Given that ORBX is an living codec, and there is already an ORBX3 in the world next year which will offer even better compression than ORBX2 (and full fp32 encoding), I am not concerned about where things stand vs. other emerging codecs – especially as HEVC will probably not be fundamentally improved upon for a decade or more, just as H.264 got stale from 2003-2013.”
I think you should be. Everyone else will be making that comparison. If you look at sites like https://www.streamingmedia.com, HEVC seems to be building mindshare as the heir apparent to H.264. A lot of the people who read those sites are video encoding consultants and they’ll be telling their clients the same thing. Some of that’s to do with video consultants not necessarily being very technical, some of it’s to do with them going with what they know, some of it’s to do with H.265 feeling like a “safe” transition if only because the name is similar, some of it’s to do with them simply not caring about any kind of “open web” ideals. None of that is a criticism of them, it’s just how they think.
They’re willing to excuse H.265’s current lack of support in hardware and software because the perception is that that’s all on the way and that H.265 offers significant benefits over H.264. VP8 wasn’t (and probably VP9 won’t be) treated the same way because it wasn’t perceived to offer anything over H.264. But to immediately contradict myself, my sense is that most of them are still of the “H.264 is good enough for me” school of thought. They’re right to be. H.264 does work almost everywhere these days, in one profile or another.
I imagine that from the business side OTOY is looking at a smaller number of bigger deals as the future for ORBX so to some extent the broader video encoding world doesn’t matter, but none of these video guys are thinking about ORBX as a future option for video streaming. And they won’t until they can see something in it for them. It’s why comparisons to all other codec options matter.
Jules wrote: “the goal we have of displacing H.264 hold on web and home video”
Brendan wrote: “That’s how codec development should proceed, not via costly custom ASIC ‘all-in’ bets.”
I agree with these goals and sentiments, it’s just that the bets have already been made. H.264 is going to be around for a long time because it’s built-in to lots and lots of phones and TVs and blu-ray players. I won’t be replacing my TV for years and it will never play ORBX streams (or H.265 or VP9 or Daala). All new codecs face the same problem and H.264’s big advantage is that it got there first as the codec that works well across broadcast, video discs and the Internet. That entrenchment will pay dividends for years to come. ORBX won’t displace H.264 any time soon, but it can maybe live along side it.
@Jim: indeed H.264 is around for the long haul. The goal with ORBX is not to replace it, rather to get out of the infernal there-can-be-only-one-h/w-codec rat race.
/be
@Jim: I added an UPDATE to get people reading your comment. Agree @sethr missed something in his CNET piece, but his intentions were all good. I’ll communicate your feedback to him. Thanks,
/be
I’m sorry, but this whole ORBX thing just sounds incredibly shady. Reading the whitepaper linked above makes about a billion alarm bells ring in my head, because the “Features” section is basically promising amazing quality coupled with faster encoding and decoding speeds under the most restrictive conditions possible (real-time low-latency encoding and decoding), and I guess the whole thing is supposed to be patent-free to boot since it’ll be free to use? You can probably see where I’m coming from by being incredibly skeptical about the whole deal.
It doesn’t help that the “comparisons” in the whitepaper are completely useless because they don’t give any support for a lot of the amazing claims made in the whitepaper, like the supposedly faster encoding and decoding speeds.
What makes them even more useless is that the results can’t be replicated or even verified by third parties, as no exact x264 commandlines are provided (no, just saying you used the “slow” preset isn’t enough when you’re omitting info about the ratecontrol method), there’s no ORBX encoder that I could download and try out myself and there aren’t even any links provided to the encoded videos.
It’d be really amazing if all this was actually true and ORBX could do all the things it claims to do, but I’m not holding my breath: As it tends to go in the world of video compression, anything that sounds too good to be true usually is.
@Daiz: we shall see. This is not just a hype exercise, OTOY is in business and has to please to serve — and please Hollywood Studios, including on picture quality.
Beyond that, the approach of designing a codec to take advantage of GPU parallelism, and doing as good as or better than H.264 at lower bit rate, is not beyond the pale. Codec design did not stop suddenly.
But, we shall have to see — and I agree that evidence verifiable by anyone when the codec is open-sourced will tell.
/be
Netflix is apparently going ahead with H.265 streaming:
https://www.stuff.tv/netflix/ces-2014/how-netflix-going-bring-4k-streaming-your-tv/news
And Google is supposed to have a lot of hardware vendors signed up for VP9 support, but there was a lot of promised for support for VP8 in the past that never materialised so we’ll see:
https://hothardware.com/News/Youtube-to-Demo-4K-Video-Streaming-With-VP9-Codec-at-CES-2014/
The rather expected reaction to Google’s VP9 announcements:
https://www.streamingmedia.com/Articles/Editorial/Featured-Articles/YouTube-and-VP9-A-Made-for-Press-Release-Event-94067.aspx
Some of the criticisms are fair enough. One sentence about promised support VP8 never being delivered on almost quotes my previous post. 🙂 But the unfortunate subtext I read into the article is that “open source” and “royalty-free” is something that only granola eating hippies value and not something you care about if you’re serious about video. But maybe I’m only seeing what I expect to see. 🙂
Nevertheless, it seems to me that this is the perception of royalty-free codecs in the minds of many video people. Fairly or unfairly, VP9, ORBX and Daala will be considered lightweight options by many unless they can overcome this image.
Jim, thanks for the followups. It is vexing that the subtext (explicit in some of the comments) dismisses (as well as sometimes conflates) open source and unencumbered standards.
On the other hand that piece is pretty clear-eyed about Google’s track record and likelihood of success. Getting a second ubiquitous codec in all systems-on-chips looks nigh-impossible to me. Getting the VP9 decoder, maybe part of the encoder, onto high-end and high-margin chips does not cut it.
/be