It's gearing up to be a busy week in WebRTC-land. In particular there's the 88th IETF meeting in Vancouver, and up for discussion and hopefully finalisation is the thorny issue of WebRTC mandatory video codec(s), with the main candidates being H.264 and VP8 this time around, but with H.265 and VP9 hoving into view in the near future as well.
Quite a lot has already been written on this topic, including by me in the October 2013 report update which has recently gone out to subscription clients, plus numerous other blogs and pundits, such as the guys at WebRTChacks who came out with a great post listing the various options last week.
But then a few days ago, Cisco Systems set the cat among the pigeons with its announcement it was open-sourcing its H.264 codec and paying the MPEG-LA royalties for anyone else who wants to use it, as long as it uses it in the form of Cisco's binary distribution. For those that want to use the Cisco source code to build their own binary however, there will remain a royalty liability.
There's a ton of great analytical blog posts around already on Cisco's move, including by Tsahi, Chris Koehnke, Dave Michels, WebRTCHacks and assorted great comments from Lawrence Byrd.
I'm hoping to make some different points, especially about mobile WebRTC and non-browser implementations.
In my view WebRTC is inevitably going to divide along an important dimension:
At present, it is hard to determine whether the future balance of importance & value is 20/80, 50/50 or 80/20 around interoperability vs standalone use-cases for WebRTC. It could go either way, and will be influenced partly by the way the industry and standards are set up in the near future. My view is that standalone uses will generally be more innovative (as is already being see with datachannel uses for WebRTC), but interoperable ones probably have easier near-term revenue pathways.
Clearly, Cisco would like to tip the balance of WebRTC towards the interop-rich use-cases, especially given its large installed base of H.264 (and also SIP) end-points. Cisco's open-sourcing move improves the ease of creating interoperability-based use cases, for example corporate videoconferencing systems extending out to browsers as additional end-points. It should be noted, however, that interoperability needs more than just codec similarity - a ton of other signalling and security issues will likely mean that gateways and SBCs are still essential, even if the theoretical transcoding burden is reduced.
However, interop is not the sole motivation, important though it is - and neither, really, is the financial benefit of H.264 royalties for the patent holders. What is also important to them is restricting the growth of new, competitive greenfield services and apps, which work best as low-cost islands - based around open-source software and (ideally) completely unencumbered codecs. Because H.264 works best with the current status-quo endpoints, it also favours current status-quo vendors and business models. It also helps to lock in H.265 as the future codec of choice, rather than VP9 or Mozilla's Dalla.
This is one reason why, in Disruptive Analysis' view, IETF should not go with H.264-only as the MTI codec, but should also add VP8. Just as there are two audio codecs (G.711 and Opus), there should also be two video options, to enable a level playing field between old and new participants. And to assure a "fair fight" between VP9 and H.265 - something which again, may skew the world between open/unencumbered and royalty-bearing.
This is especially true as the MPEG-LA licencing regime is aimed very much at the traditional media industry in terms of streamed video, rather than application developers using new forms of communication. The 100k user threshold at which H.264 royalties become payable typically means a quite successful content or web company - but 100k users for a free mobile app is almost trivial - and almost certainly not profitable. The current licencing regime is heavily stacked against the small developer, and free/freemium models.
Consequently, next-generation developers will need an unencumbered video codec of one form or another - both for the desktop web, and for mobile apps.
Indeed, all the discussion about Cisco's H.264 move assumes that WebRTC remains a single, web-based standard. But the other, little-discussed dimension here is whether WebRTC use-cases will be primarily based in the browser or not. My view is that on the desktop, WebRTC will indeed be predominantly browser-based, although we'll also see it appear in some standalone applications.
But on smartphones, tablets and other devices, WebRTC will primarily be embedded into native apps, or the underlying device OS, and not necessarily the browser. That is *especially* true on iOS, unless Apple suddenly changes its mind on WebRTC support in mobile Safari.
Let me reiterate that: on mobile devices, the bulk of WebRTC will NOT appear in-browser, but instead it will be in-app, with either the developer directly integrating the various WebRTC code modules (such as codecs, DTLS, SRTP etc) into an app, or being provided with a 3rd-party API and code libraries by cloud-comms player like TokBox or Tropo.
In my newest forecasts, from the middle of 2015, there will be MORE non-browser mobile implementations of WebRTC than mobile browser devices. There will be 2.3bn mobile devices with non-browser support of WebRTC by end-2016.
That means that WebRTC on mobile (especially iOS) probably cannot easily exploit Cisco’s H.264 offer. There needs to be a way for either standalone app developers to get access to iOS’ existing H264 (and iPhone hardware acceleration is not currently allowed under present APIs) or else there needs to be “BYOcodec” with the app. In other words, either a separate (and potentially royalty-bearing) implementation of H.264, or else VP8. My expectation is that future mobile chipsets will likely support hardware acceleration of both H.264/5 and VP8/9 - although whether the OEMs like Apple provide API access is another matter.
In other words, Cisco's kind offer will be irrelevant for most mobile implementations of WebRTC. That doesn't mean Cisco is being duplicitous here - just that it cannot, by itself, influence those forms of distribution and use of H.264, especially on iOS.
For this reason, Disruptive Analysis believes that in order of preference, the IETF should choose as MTI video codec:
1) [Best] H.264 + VP8, with a clear statement of intent on H.265 + VP9 for the future
2) An old, out-of-patent codec so that there's a minimum fallback option, but each app chooses its preferred codec as it needs to
3) No mandatory video codecs at all. Fallback is voice-only, unless an app specifically negotiates video.
4) [worst] Either VP8-only or H.264-only.
[Also - I've seen a couple of mentions on Twitter about the idea of a mandatory codec selection API, rather than a codec itself. Also sounds like a good idea, but not exactly sure how it would work in practice. I'll keep an eye out]
Quite a lot has already been written on this topic, including by me in the October 2013 report update which has recently gone out to subscription clients, plus numerous other blogs and pundits, such as the guys at WebRTChacks who came out with a great post listing the various options last week.
But then a few days ago, Cisco Systems set the cat among the pigeons with its announcement it was open-sourcing its H.264 codec and paying the MPEG-LA royalties for anyone else who wants to use it, as long as it uses it in the form of Cisco's binary distribution. For those that want to use the Cisco source code to build their own binary however, there will remain a royalty liability.
There's a ton of great analytical blog posts around already on Cisco's move, including by Tsahi, Chris Koehnke, Dave Michels, WebRTCHacks and assorted great comments from Lawrence Byrd.
I'm hoping to make some different points, especially about mobile WebRTC and non-browser implementations.
In my view WebRTC is inevitably going to divide along an important dimension:
- Use-cases that need interoperability with “other stuff” (eg existing videoconferencing, enterprise UC systems, telco SS7/IMS etc)
- Use-cases that are perfectly fine as standalone islands (eg a karaoke app, business walkie-talkie apps, or a customised video-chat service). Some of these will be P2P, but many will be server-based as well.
At present, it is hard to determine whether the future balance of importance & value is 20/80, 50/50 or 80/20 around interoperability vs standalone use-cases for WebRTC. It could go either way, and will be influenced partly by the way the industry and standards are set up in the near future. My view is that standalone uses will generally be more innovative (as is already being see with datachannel uses for WebRTC), but interoperable ones probably have easier near-term revenue pathways.
Clearly, Cisco would like to tip the balance of WebRTC towards the interop-rich use-cases, especially given its large installed base of H.264 (and also SIP) end-points. Cisco's open-sourcing move improves the ease of creating interoperability-based use cases, for example corporate videoconferencing systems extending out to browsers as additional end-points. It should be noted, however, that interoperability needs more than just codec similarity - a ton of other signalling and security issues will likely mean that gateways and SBCs are still essential, even if the theoretical transcoding burden is reduced.
However, interop is not the sole motivation, important though it is - and neither, really, is the financial benefit of H.264 royalties for the patent holders. What is also important to them is restricting the growth of new, competitive greenfield services and apps, which work best as low-cost islands - based around open-source software and (ideally) completely unencumbered codecs. Because H.264 works best with the current status-quo endpoints, it also favours current status-quo vendors and business models. It also helps to lock in H.265 as the future codec of choice, rather than VP9 or Mozilla's Dalla.
This is one reason why, in Disruptive Analysis' view, IETF should not go with H.264-only as the MTI codec, but should also add VP8. Just as there are two audio codecs (G.711 and Opus), there should also be two video options, to enable a level playing field between old and new participants. And to assure a "fair fight" between VP9 and H.265 - something which again, may skew the world between open/unencumbered and royalty-bearing.
This is especially true as the MPEG-LA licencing regime is aimed very much at the traditional media industry in terms of streamed video, rather than application developers using new forms of communication. The 100k user threshold at which H.264 royalties become payable typically means a quite successful content or web company - but 100k users for a free mobile app is almost trivial - and almost certainly not profitable. The current licencing regime is heavily stacked against the small developer, and free/freemium models.
Consequently, next-generation developers will need an unencumbered video codec of one form or another - both for the desktop web, and for mobile apps.
Indeed, all the discussion about Cisco's H.264 move assumes that WebRTC remains a single, web-based standard. But the other, little-discussed dimension here is whether WebRTC use-cases will be primarily based in the browser or not. My view is that on the desktop, WebRTC will indeed be predominantly browser-based, although we'll also see it appear in some standalone applications.
But on smartphones, tablets and other devices, WebRTC will primarily be embedded into native apps, or the underlying device OS, and not necessarily the browser. That is *especially* true on iOS, unless Apple suddenly changes its mind on WebRTC support in mobile Safari.
Let me reiterate that: on mobile devices, the bulk of WebRTC will NOT appear in-browser, but instead it will be in-app, with either the developer directly integrating the various WebRTC code modules (such as codecs, DTLS, SRTP etc) into an app, or being provided with a 3rd-party API and code libraries by cloud-comms player like TokBox or Tropo.
In my newest forecasts, from the middle of 2015, there will be MORE non-browser mobile implementations of WebRTC than mobile browser devices. There will be 2.3bn mobile devices with non-browser support of WebRTC by end-2016.
That means that WebRTC on mobile (especially iOS) probably cannot easily exploit Cisco’s H.264 offer. There needs to be a way for either standalone app developers to get access to iOS’ existing H264 (and iPhone hardware acceleration is not currently allowed under present APIs) or else there needs to be “BYOcodec” with the app. In other words, either a separate (and potentially royalty-bearing) implementation of H.264, or else VP8. My expectation is that future mobile chipsets will likely support hardware acceleration of both H.264/5 and VP8/9 - although whether the OEMs like Apple provide API access is another matter.
In other words, Cisco's kind offer will be irrelevant for most mobile implementations of WebRTC. That doesn't mean Cisco is being duplicitous here - just that it cannot, by itself, influence those forms of distribution and use of H.264, especially on iOS.
For this reason, Disruptive Analysis believes that in order of preference, the IETF should choose as MTI video codec:
1) [Best] H.264 + VP8, with a clear statement of intent on H.265 + VP9 for the future
2) An old, out-of-patent codec so that there's a minimum fallback option, but each app chooses its preferred codec as it needs to
3) No mandatory video codecs at all. Fallback is voice-only, unless an app specifically negotiates video.
4) [worst] Either VP8-only or H.264-only.
[Also - I've seen a couple of mentions on Twitter about the idea of a mandatory codec selection API, rather than a codec itself. Also sounds like a good idea, but not exactly sure how it would work in practice. I'll keep an eye out]
You have been the voice arguing about the long term dominance of embedded mobile WebRTC for a long while, and maybe the rest of us are starting to catch up! For me this means that the providers of mobile devices need to step up and provide API-accessible hardware-accelerated video codecs with both encode and decode accessible, that are already hidden within their platforms. And they should follow Cisco's lead in dealing with the licensing for their own platforms (and not expecting someone else to do this for them). It is crazy that we are all currently discussing how to "bring your own codec". So let's ask Apple, Microsoft/Nokia, Samsung and others what THEY will be doing to open up this market opportunity? Some of these are very rich companies already plus will also be the beneficiaries of an increasingly mobile real-time world.
ReplyDeleteUnknown=@LawrencByrd btw (these comment systems..).
ReplyDelete