Deep inspection of Allot's mobile data trends report

Quite a few technology vendors put out interesting research reports on mobile data - the best-known probably being Cisco's VNI data and forecasts, which gets cited by about half the rest of the industry.

However, a number of the smaller DPI and policy vendors (also WiFi specialists) also put out papers and reports, sometimes based on observed data from their own implementations, and sometimes from commissioned surveys.

(I've got absolutely no problem with this in principle - I've done various reports and papers for companies myself, although typically they've been for those wanting a "thought leadership" position associated with my often-contrarian opinions and analyses).

Clearly, all these reports are for principally for the purpose of raising awareness, and acting as marketing vehicles, by providing interesting newsworthy soundbites, and grabbing the attention of network purchasing folk at operators. But it's worth scrutinising them for what they say, what they don't say, and the methodology/assumptions involved.

One of these companies is Allot Communications, which has issued a series of reports on mobile data bandwidth use, applications and so forth. It's just published its H1 2011 report, downloadable here.

There's some good stuff in there, but also plenty which raises questions.

• The source of the data for the report is Allot's own installed base of network elements, spanning "networks representing more than 250 million subscribers". It's entirely unclear which operators these are. That's critical, because it determines if this is a representative sample of the world's 5 billion or so subsciptions, or if it's somehow skewed because of particular operators' or countries' specific local circumstances. It's also unclear how many of the 250m are active data users at all - my reading is that's the "potential" reach of those networks, not the current user base.This is all critical, because if the survey is based on (say) developing-world operators, you'd expect to see a much higher overall growth rate for data than in mature markets.
• The most glaring omission is any reference to the volume of traffic from laptops (3G dongles) versus smartphones or other devices.  This is hugely important in terms of interpreting the other statistics, as many dongles are sold as alternatives to fixed broadband, so you expect a broadly similar usage profile. It may well be that a large part of VoIP, P2P and mobile video streaming is consumed on PCs, which most operators cannot change easily - it's pretty hard to say that a USB modem service is "just like ADSL, except you can't use Skype. Or YouTube in HD" and remain competitive.
• However, an important guide is filesharing. Generally, smartphones aren't significantly responsible for P2P traffic as far as I know. That suggests that the bulk of the 29% will come from PCs - and therefore, so will much of the video and web browsing, as almost nobody *just* uses PC and dongle to swap files. (As evidenced by Allot's chart on *fixed* broadband traffic) In other words, I expect that the contribution of PC mobile broadband is hugely skewing the overall study. I'm going to call out Allot and say it's trying to avoid this discussion - there is no mention of the word laptop, notebook, PC, modem or dongle in the whole document. The word "smartphone" appears 5 times.
• There are no absolutes in terms of MB or GB, or in terms of actual numbers of unique users. So it's impossible to tell if growth is coming from more subscribers or more use per subscriber. I suspect that the average might actually be tending down as we see a shift from dongles to smartphones, and as late-adopters start getting smartphones.
• It's unclear whether *all* data traffic gets funnelled through the Allot box in those networks. Does some get siphoned off "in front" of the box? (eg telco-hosted data services, BlackBerry traffic or corporate VPNs) Does some get injected in deeper in the network (eg with a CDN)? Where there are video compression / optimisation boxes or caches, is the data showing the compressed or uncompressed amounts of data? Are there any proprietary direct-tunnel or offload solutions involved that bypass the core network?
• The report misuses the term "application" - video is not "an application" but a traffic type. An application (at a user level) can involve several different traffic types, for instance a web page with an embedded video advert or an audio plug-in.
• Application-aware charging is something that most DPI vendors are huge fans of (unsurprisingly, as it typically needs DPI boxes), but which I'm a huge critic of. The study is based on analysis of vendors' stated policies or tariffs on the web, but it's unclear exactly what "application" means here - for example, many operators have very different charging for M2M data devices and applications than for smartphones or dongles. It's not clear from the Allot survey that the reported 32% of operators using app-aware charging are doing this with DPI, rather than (say) using a separate APN for BlackBerries or Facebook Zero or whatever. (It's worth noting that essentially *all* operators zero-rate internal data traffic used for device management)
• Some of the definitions are pretty woolly. So-called VoIP traffic also includes video communications (eg Skype) and presumably also the associated IM and advertising data, although those are small at present. Given that a huge % of Skype calls are video-based (from laptops!) that's rather important. So we have the strange situation that some of the most-used mobile IM platforms - Skype, Facebook Chat and BlackBerry Messenger - don't appear in the chart at all, although BBM's absence could be attributed to a laptop-centric overall sample.
• The word "signalling" doesn't appear at all
• Neither does the word "encryption" or HTTPS - both of which are becoming increasingly important, and which are essentially opaque to most forms of DPI

Overall, there's some good data points there, but a "deep inspection" suggests that there's rather a lot that's not being said. In particular, the downplaying of the role of PC mobile broadband seems deliberate. Allot is also very keen to talk up "personalisation" in terms of "app-aware charging", but seems to have been pretty selective with its evidence to support its assertions.

Edit: it's just struck me that this is a piece of analysis that is based on interpretation and inferences, rather than direct collaboration or discussion with the content provider. Just like DPI and application-aware networking, in other words. (My self-determined acceptable rate for "false positives" on a piece of this type is 10%. If I've got more than that amount wrong, I'll do a re-write or retraction. I've yet to see a DPI vendor give a false-positive threshold).