Recently, I've been becoming increasingly aware that the mobile/wireless industry often plays fast-and-loose with statistics. It does this both to appear good externally (to investors, commentators, regulators, and scaring competitors) and as a form of "positive affirmation" internally (OK, OK, self denial....) when it is confronted with unpleasant truths.
Before I get a flood of emails, obviously the mobile industry isn't the only one that plays with numbers to put a positive spin on things. The marketing industry's use of surveys does the same thing across every segment from cars to chocolate to (I imagine) industrial waste disposal. Most government bodies do the same as well (step forward Tony, Gordon & Ken, in particular.....)
I suspect that this will be an ongoing theme, but I'd like to try and compile a "top 10" list of numeric fudges. Let's see:
1) Subscribers = Users
- Fudge: multiple phone/SIM ownership by many people
2) Overall ARPU numbers
- Fudge: exclusion of rebates, and similar repayments that come out as "costs" rather than lost revenue
3) Data ARPU (1)
- Fudge: Inclusion of revenue from enterprise & data-only devices (laptop cards, M2M modules, Blackberries) aggregated in non-SMS Data ARPU, and then using the figures to suggest that consumer use of content/MMS/etc is rising faster than it actually is
4) Data ARPU (2)
- Fudge: Revenue allocation of bundling. So, I get 250 minutes + 500 texts per month for £30 (+ video / MMS / whatever). How do they decide that "inside the bundle" 1 min = 5p / 1 text = 10p ... or 1 min = 3p / 1 text = 14p ? How is this audited? How much "arbitrariness" is there when operators report "Data ARPU was 19%"?
5) Coverage %
- Fudge: it's outdoor coverage, bizarrely using "percentage of population covered". Which is strange, as I always thought that most of the population lives indoors.
6) Radio downlink speeds (for 3G, or WiFi, or WiMAX or whatever)
- Fudge: what's the aggregate capacity of the cell? so, how many concurrent users of your 1Mbit/s HSDPA saturate the base station, or its backhaul?
7) Comparisons mobile vs. broadband
- Fudge: comparing apples & pears - people vs. households. Sometime useful, but often irrelevant as both are important metrics for different reasons.
8) Comparisons of mobile phones vs PCs
- Fudge: no, most people's "computing experience" won't be on phones, even in developing countries. Have you been to many developing countries, Mr Marketing VP? Seen kids packed into in Internet shops using MSN & doing their homework? Stop using the (equally spurious) home PC penetration figures & understand the real world
9) Early mobile application market growth numbers
- Fudge: often include substantial "internal" use by the mobile industry itself - application developers, competitors, journalists, testing (including automated network probes) etc, rather than "proper customers"
10) Smartphone market size
- Fudge: Includes a huge number of "closed" operator-customised phones developed on theoretically open OSs (eg DoCoMo FOMA use of Symbian & Linux) which are locked-down for 3rd-party software, plus a vast number of Nokia S60-based devices bought by people who neither know nor care that their device has an open "smart" OS. The proportion of smartphones bought by individuals who knowingly choose an "open" & flexible software platform is probably less than 20%.
I'm sure there's loads more of these type of things.... any suggestions?
Dean Bubley's Disruptive Wireless: Thought-leading wireless industry analysis
Mobile operators showing mobile voice as % of total voice traffic claiming growing trend towards FMS.
ReplyDeleteFudge: Yes, FMS is happening, but not as fast as these figures suggest. These tends to exclude all those minutes that are indeed migrating off the PSTN, but going into VoIP. Take Norway as an example.
Fixed voice traffic (PSTN) lost 7.8 billion minutes from 2004 to 2005, but mobile traffic only went up by 1.5 billion over same period.
Where are the missing 6 billion+ minutes?
Well, fudge is used also for VoIP: VoIP providers typically show VoIP penetration as % of broadband connections (e.g. in Norway...). So basically showing a growth which is reflects the growth of something else. VoIP as PSTN replacement is less than 2% of a slowly shrinking fixed line market (globally).
ReplyDeleteAgreed that the fudge factors 1-10 are widely used and obscure truth.
Nevertheless the trends remain true in many cases. For example, 2 B cellular users is still a lot more than about 1 B Internet users (not households) globally.
And anyway, current market figures cannot prove or disprove disruptions such as how many consumers will do "mobile computing" and what kind of devices they will want to use.
8) Comparisons of mobile phones vs PCs
ReplyDelete- Fudge: no, most people's "computing experience" won't be on phones, even in developing countries. Have you been to many developing countries, Mr Marketing VP? Seen kids packed into in Internet shops using MSN & doing their homework? Stop using the (equally spurious) home PC penetration figures & understand the real world
Ok, probably 'most' people is wrong terminology, but atleast a lot of people in the developing countries would be having their first 'computational' experience on a mobile phone. Ofcourse, they may not be able to identify it as their 'first stint with a pc'.
Consider this, in India (where I am from) there are many many more mobile phone users than computer users. Even if you count those who go to cyber cafes to use a comp...or the ones who get to use one in workplace / school. Many of these mobile users have never ever used a computer..heck many of them would not be able to pronounce the phrase 'computational experience'.
Need more examples - my parents learnt to use a mobile phone before a computer. The plumber in my house would not be able to differentiate between a computer display and a regular television set. The local rickshaw driver would not know what a computer is. Many kids, many from these families, would not have had a computer at school or home, but have operated their parent's mobile phone.
What term would you give to the 'experience' of these people, when they play their first video game on the mobile phone?
Or when they download a new ringtone and 'install' it?
Or, technolgoy and price permitting, take a picture from a camera-phone in the future and store it, review it (sending it may still be complicated ;-)
Hi Dean and Disruptive Wireless readers
ReplyDeleteGREAT POSTING !!!
And I feel a pang of pain, I've been guilty of it too many times myself. I keep talking of mobile phone users, while I know fully well its subscrptions. By my analysis currently 9.7% of all subscribers, and over 20% of European mobile phone subscribers have two subscriptions (or more).
We need to be diligent. We need to "police" the sloppy numbers.
I would make one observation, however on ARPU and subscriptions (and users). When the mobile operators report an ARPU figure, they take total revenues divided by total subscribers.
Now, across the national subscriber base say in Europe, that means that a Vodafone counts an average 30 Euro charge for one subscriber. That same subscriber is also on T-Mobile, who also reports an average 30 Euro ARPU.
NOTE - this is consistent in the industry - in reality those with two subscriptions do actually deliver "two times ARPU" in total revenues. Some are poor and actually have low-end phone bills on both phones (but can have high incoming calls, which could explain the ARPU). But others are those with astronomical ARPUs, say one unlimited use phone from the employer, and another one with average usage on the personal phone.
So we need to also understand that the multiple subscriptions do allow for a "hidden" bonus of revenues. Its not that if we eliminate double-counting of subcriptions our total revenues would shrink. Rather, that INSPITE of double subscriptions, the average global ARPU is 28 dollars, etc...
Great posting Dean. Looking forward to Part 2...
Tomi Ahonen :-)
Fudge # 11: to count as an active subscriber someone who´s not active at all, just sticking into the list of subscribers because of other reasons (default, fraud, etc.).
ReplyDeleteAs an example of this fudge, just look back to few days ago, when Vivo, the biggest Brazilian mobile operator and part of the Telefonica group, decided to deduct 1,8 Million subscribers (yep, you read well, 1,8 Million subscribers) from its base of more than 30 Million subscribers. Really a bad time to be a shareholder of Vivo.
Portuguese readers find details here: http://www.estadao.com.br/ultimas/economia/noticias/2006/jul/21/301.htm
part of 1)
ReplyDeleteChurn = subscribers
Subscribers who churn in a period are counted both by the carrier they migrate to, and the one they come from... inflating tot subscriber numbers in a market by 15-20% or more.
the missing 6 billion minues (from the first post) went to mobile email. my usage went from 4000 minutes per month on my cell phone to less than 1000, but my emails are up over 100 daily. email is a nice papertrail for client interaction, and it eliminates the he said / she said senario.
ReplyDeleteRE: churn. No, churned subs are not counted twice. They are only counted by the receiving carrier. The outgoing carrier counts them as churn (deactivations). The outgoing still pays their invoice though so it counts as churn
ReplyDeleteSorry, last entenc should read "The outgoing still pays their invoice though so it counts as revenue".
ReplyDeleteI think you may have to change the list to 'Top 20'..... Howabout:
ReplyDeleteCarriers counting billed revenue from 3rd party aggregators as ARPU, without subtracting the payouts they are obligated to provide the 3rd party aggregator.
1. Carrier bills and collects from the end-user $5.00 for 3rd party content
2. Carrier pays $3.00 to 3rd party
3. Carrier counts $5.00 for revenue instead of $2.00
this is never clear in reported data revenue figures.
Most of the complaint about "fudging" sounds like whining to me. I work in wireless and I would love to have perfect data. Unfortunately I do not; so I work with what we are able to mine accurately. The published numbers are virtually all aimed at the investment community so we are able to accurately provide revenue, subscription, and cost information...those things that drive business models the most.
ReplyDeleteOther things, like trying to examine traffic to determine the type of activity (voip, email, media, etc) are difficult and expensive on the one hand and raise privacy concerns on the other so they are either:
a) not worth it, or
b) too valuable to make freely available.
Additionally that still leaves the challenge of deciding whether that 10MB of VOIP traffic should count as data revenue or voice revenue.
So I would be hard pressed to call it "fudging", which implies an effort to conceal data. "Best effort" maybe, or "caveat emptor" as is everything else in life.
A lot of data we just don't have.
Jim, ARPU is "average REVENUE per user" (user is a single device). It includes no cost components by definition, how can that be a fudge?
ReplyDeleteNo-one reports "profit-per-unit" or even "gross margin per unit". That is sensitive competitive information.
Mobile data including P2P SMS traffic.
ReplyDeleteIncluding MMS traffic within SMS totals.
Talking about growth of MMS as a per centage but not the actual number of messages.
Mike
www.160characters.org
Excellent post. I linked to it from my Newsletter: http://www.fiercedeveloper.com/story/fudging-the-numbers-in-wireless-reports/2006-07-25
ReplyDeleteCan't wait for part 2.
Anything not under some GAAP will be fudged. It's natural for a company to make data look good, and if a dodgy definition can be used, then it will be.
ReplyDeleteThe fault should be with GAAP metrics definers.
A report in 2003 (?) from Yankee Group called "Of Apples & Oranges" looked at "Total ARPU" definitions across US carriers. There was not one pair of companies with the same defintion.
Revenue items which are sometimes includes or not: roaming revenue from non-subscribers; mast sharing revenue; handset & accessories revenue; fines [e.g. late payments]; activation revenue; BOBO services [e.g. ringtones]. Yankee showed (as I remember) that 30% can be added to ARPU depending on the definition.
It's the job of the analyst community to filter the bull--that's why they get paid.
Data ARPU Fudge number #3: Some operators include voice mail in their Data ARPU numbers, justifying it as a call to a database (not a person), therefore it is data. So voice mail surcharges and per-minute charges for calls to voice mail boxes are included in data ARPU.
ReplyDeleteAs for subscriptions vs. users, I'm in the mobile industry and currently have 5 active mobile subscriptions. Bot for now, at least, there is just one of me.
Does anyone else find it ironic that an ex-market analyst and equity analyst should accuse the industry of fudging numbers? Pot, meet kettle! Kettle, meet pot!
ReplyDeleteThanks all for these comments. I'm working on a "Part 2"....
ReplyDeleteA couple of responses:
Worma - I would not describe watching a mobile video or downloading a ringtone as a "computing experience". On that basis you could also argue that "setting the programme on a washing machine", "getting cash from an ATM machine" or "watching a DVD" was also a "computing experience".
Anonymous - yes, I know good data is hard to find. But I absolutely do believe that what data actually is available is often indeed presented in a fashion "which implies an effort to conceal"
Dan - Pot/Kettle/Black, well firstly, what's with the "ex" market analyst? Given you've entered your identifying web address as my old firm Datamonitor, I'm a bit sad to see that as a fellow researcher, you haven't bothered to understand my current business, nor my history of preferring to be "right" or at least cogently-argued rather than "just getting some numbers out".
Worma - I would not describe watching a mobile video or downloading a ringtone as a "computing experience". On that basis you could also argue that "setting the programme on a washing machine", "getting cash from an ATM machine" or "watching a DVD" was also a "computing experience".
ReplyDeleteDean - Now we're playing with words, aren't we! I believe you used the phrase 'computing experience' to refer to the first technological stint that the end users would have, to achieve certain goals. And here I mean requirements that are 'met' by using the computing powers of the computer (plus the internet info stream)
And it is such needs, that I feel, would be covered better by mobile phones - whether it is the first experience or the subsequent ones.
If this was not close to your intention, then I stand corrected.
This comment has been removed by a blog administrator.
ReplyDeleteI also work in the wireless idustry. I have for over 15 years. I can positively say it is all one big fudge. I have first hand data that proves well thought out "fudges" in order to report stronger data than what actually exists. Rather than go through the entire list, please feel free to ask for specifics. The unfortunate issue here is when I have raised these concerns internally they are ignored.
ReplyDeleteFudging numbers! Ha. WOrking with numbers inside an operator is an art. That's the supply side. Then there's the demand side - all sorts of customers - management who need to stop the operator going bankrupt; internal groups who need the info to justify extra headcount in customer service; analysts who need anything to go and make predictions; competitors who armed with the best info you have may be used against you; investors; stock exchange; etc etc.
ReplyDeleteSome difficult ones to produce are:
"active users" say of voicemail. How does one define this without fudging? Is it all those with mailboxes who have received at least one voicemail in the last three months, and who has logged in (perhaps without listening to the voicemail?)? It once took me 24 hours to download just a small amount of voicemail logfiles for data manipulation, and at a time when a datawarehouse change requested pipeline was out to 18 months.
What about those who visit an operator on-deck portal. Is once enough in 6 months?
Am against numbers however determined without reporting assumptions beside them. Unfortunately these are lost when they hit the public domain sometimes.