MEC and network-edge computing is overhyped and underpowered

I keep hearing that Edge Computing is the next big thing - and specifically, in-network edge computing models such as MEC. (See here for a list of all the different types of "edge"). 

I hear it from network vendors, telcos, some consultants, blockchain-based startups and others. But, oddly, very rarely from developers of applications or devices.

My view is that it's important, but it's also being overhyped. Network-edge computing will only ever be a small slice of the overall cloud and computing domain. And because it's small, it will likely be an addition to (and integrated with) web-scale cloud platforms. We are very unlikely to see edge-first providers become "the next Amazon AWS, only distributed".

Why do I think it will be small? Because I've been looking at it through a different lens to most: power. It's a metric used by those at the top- and bottom ends of the computing industry, but only rarely by those in the middle, such as network owners. This means they're ignoring a couple of orders of magnitude.

(This is a long post. You might want to grab a coffee first....)


How many zeroes?

Cloud computing involves huge numbers. There are many metrics that you can use - numbers of servers, processors, standard-sized equipment racks, floorspace and so on. But the figure that gets used most among data-centre folk is probably power consumption in watts, or more commonly here kW, MW & GW. (Yes, it's a lower-case k for kilo). 

Power is useful, as it covers the needs not just of compute CPUs and GPUs, but also storage and networking elements in data centres. It's not perfect, but given that organising and analysing information is ultimately about energy it's a valid, top-level metric. [Hey, I've got a degree in physics, not engineering. Helloooo, thermodynamics & entropy!]

Roughly speaking, the world's big data centres have a total power consumption of about 100GW. A typical one might have a capacity of 30MW, but a number of the world's largest data centres already use over 100MW individually, and there are enormous plans for locations with 600MW or even 1GW (link). No, they're not all running at full power, all the time - but that's true of any computing platform.

This growth is partly driven by an increase in the number of servers and equipment racks needed (hence growing floor-space for these buildings). But it also reflects power consumption for each server, as chips get more powerful. Most equipment racks use 3-5kW of power, but some can go as high as 20kW if that power - and cooling - is available.

So, to power "the cloud" needs 100GW, a figure that is continuining to grow rapidly. We are also seeing a rise in smaller, regional data-centres in second- and third-tier cities. Companies and governments often have private data-centres as well. These vary quite a bit, but 1-5MW is a reasonable benchmark.


How many decimal places?

At the other end of the computing power spectrum, are devices, and the components inside them. Especially for battery-powered devices, managing the power-budget down to watts or milliwatts is critical. This is the "device edge".

• Sensors might use less than 10mW when idle & 100mW when actively processing data
• A Raspberry Pi might use 0.5W
• A smartphone processor might use 1-3W
• An IoT gateway (controlling various local devices) might be 5-10W
• A laptop might draw 50W
• A decent crypto mining rig might use 1kW

New innovations are pushing the boundaries. Some researchers are working on sub-milliwatt vision processors (link). ARM has designs able to run machine-learning algorithms on very low-powered devices.

But perhaps the most interesting "device edge" is the future top-end Nvidia Pegasus board, aimed at self-driving vehicles. It is a 500W supercomputer. That might sound a lot, but it's still less than 1% of the engine power on most cars. A top-end Tesla P100D puts over 500kW to the wheels in "ludicrous mode", or 1000x that figure. Cars' aircon might use 2kW, to give context.

Of course, all of these device-edge computing platforms are numerous. There are billions of phones, and hundreds of millions of vehicles and PCs. Potentially, we'll get 10s of billions of sensors. Most aren't coordinated, though. 


And in the middle?

So we have milliwatts at one end of distributed computing, and gigawatts at the other, from device to cloud.

So what about the middle, where the network lives?

There are many companies talking about MEC (multi-access edge computing) and fog-computing products, with servers designed to run at cellular base stations, network aggregation points, and also in fixed-network nodes and elsewhere. 

Some are "micro-data-centres" capable of holding a few racks of servers near the largest cell towers. The very largest might be 50kW shipping-container sized units, but those will be pretty rare and will obviously need a dedicated power supply.

It's worth noting here that a typical macro-cell tower might have a power supply of 1-2kW. So if we consider that maybe 10% could be dedicated to a compute platform rather than the radio (a generous assumption), we get 100-200W, in theory. Or in other words, a cell tower edge-node will be less than half as powerful as a single car's computer.

Others are smaller server units, intended to hook into cellular small-cells, home gateways, cable street-side cabinets or enterprise "white boxes". For these, 10-30W is more reasonable.

Imagine the year 2023

Let's think 5 years ahead. By then, there could probably be 150GW of large-scale data centres, plus a decent number of midsize regional data-centres, plus private enterprise facilities.

And we could have 10 billion phones, PCs, tablets & other small end-points contributing to a distributed edge, although obviously they will spend a lot of time in idle-mode. We might also have 10 million almost-autonomous vehicles, with a lot of compute, even if they're not fully self-driving. 

Now, imagine we have a very-bullish 10 million "deep" network-compute nodes, at cell sites large and small, built into WiFi APs or controllers, and perhaps in cable/fixed streetside cabinets. They will likely have power ratings between 10W and 300W, although the largest will be numerically few in number. Choose 100W on average, for a simpler calculation. (Frankly, this is a generous forecast, but let's run with it for now).

And let's add in 20,000 container-sized 50kW units, or repurposed central-offices-as-datacentres, as well. (Also generous)

In other words, we might end up with:

150GW large data centres
50GW regional and corporate data centres
20,000x 50kW = 1GW big/aggregation-point "network-edge"
10m x 100W = 1GW "deep" network-edge nodes
1bn x 50W = 50GW of PCs
10bn x 1W = 10GW "small" device edge compute nodes
10m x 500W = 5GW of in-vehicle compute nodes
10bn x 100mW = 1GW of sensors & low-end devices

Now admittedly this is a very crude analysis. And a lot of devices will be running idle most of the time, and may need to offload functions to save battery power. Laptops are often switched off entirely. But equally, network-edge computers won't be running at 100%, 24x7 either.


The 1% edge

So at a rough, order-of-magnitude level, we can see that the total realistic "network edge", with optimistic assumptions, will account for less than 1% of total aggregate compute capability. And with more pessimistic assumptions, it might easily be just 0.1%. 

Any more will simply not be possible to power, unless there are large-scale upgrades to the electricity supply to network infrastructure - installed at the same time as backhaul upgrades for 5G, or deployment of FTTH. (And unlike copper, fibre can't even power small devices on its own). And haven't seen announcements of any telcos building hydroelectric power stations anywhere.

Decentralised, blockchain-based edge "fogs" are unlikely to really solve this problem either, even if they also use decentralised, blockchain-based power supply and management.

Now it could be argued that this 0.1-1% of computing workloads will be of such pivotal importance, that they will bring everything else into their orbit and indirect control. Could the "edge" really be the new frontier? 

I think not.

In reality, the reverse is more likely. Either device-based applications will selectively offload certain workloads to the network, or the webscale clouds will distribute certain functions. Yes, there will be some counter-examples, where the network-edge is the control point for certain verticals or applications - I think some security functions make sense, for instance, as well as an evolution of today's CDNs. But will IoT management, or AI, be concentrated in these edge nodes? It seems improbable.


Conclusion & TL:DR

In-network edge-computing architectures, such as MEC, will become more important. There are various interesting use-cases. But despite that, they will struggle to live up to the hype. 

There will be almost no applications that run *only* in the network-edge - it’ll be used just for specific workloads or microservices, as a subset of a broader multi-tier application. The main compute heavy-lifting will be done on-device, or on-cloud. As such, collaboration between edge-compute providers and industry/webscale cloud will be needed, as the network-edge will only be a component in a bigger solution, and will only very rarely be the most important component. 

One thing is definite: mobile operators won’t become distributed quasi-Amazons, running image-processing for all nearby cars or industry 4.0 robots in their networks, linked via 5G. 

Yes, MEC nodes could host Amazon Greengrass or other functions on a wholesale basis, but few developers will want to write directly to telcos' distributed-cloud APIs on a standalone basis, with or without network-slicing or 5G QoS mechanisms.

Indeed, this landscape of compute resource may throw up some unintended consequences. Ironically, it seems more likely that a future car's hefty computer, and abundant local power, could be used to offload tasks from the network, rather than vice versa.

Comments and feedback are very welcome. I'm aware I've made many assumptions here, and will doubtless generate various comments and detailed responses, either on my blog or LinkedIn posts. I haven't seen an "end to end" analysis of compute power before - if there's any tweaks to my back-of-envelope calculations, I'd welcome suggestions. If you'd like to contact me about projects or speaking engagements, I can be reached via information at disruptive-analysis dot com.

TelcoFuturism: Will AI & machine-learning kill the need for network QoS?

Following on from my introductory post about TelcoFuturism (link), this is a forward-looking "what if?" scenario. It arises from one impending technology intersection - the crossover between network policy-management, real-time applications (especially voice & video) and machine-learning/artificial intelligence (AI)

One of the biggest clichés in telecoms is that every new network technology allows the creation of special "quality of service" characteristics, that potentially enable new, revenue-generating, differentiated services. But while QoS and application-based traffic-engineering certainly is useful in some contexts - for example, managed IPTV on home broadband lines, or prioritisation of specific data on enterprise networks - its applicability to a wider audience remains unproven. 

In particular, end-to-end QoS on the public Internet, paid-for by application or content providers and enforced by DPI and in-network policy engines, remains a fantasy. Not only does Net Neutrality legislation prohibit it in many cases, but the concept is an undesirable and unworkable fallacy to begin with. 

App-specific QoS doesn't work technically on most shared networks (ask colleague Martin Geddes, who'll enlighten you about the maths of contention-management). There's no way to coordinate it all the way from server-to-user-access. While CDNs and maybe future mobile edge nodes might help a bit, that's only a mid-point, for certain applications. On mobile devices, the user is regularly using one of millions of 3rd-party WiFi access points, over which the app-provider has no control, and usually no knowledge. The billing and assurance systems aren't good enough to charge for QoS and ensure it was delivered as promised. Different apps behave differently on different devices and OS, and there's no native APIs for developers to request network QoS anyway. And increasing use of end-to-end encryption makes it really hard to separate out the packets for each application, without a man-in-the-middle.

There's also another big problem: network quality and performance isn't just about throughput, packet-loss, latency or jitter. It's also about availablility - is the network working at all? Or has someone cut a fibre, misconfigured a switch, or just not put radio coverage in the valley or tunnel or basement you're in? If you fall off of 4G coverage back to 3G or 2G, no amount of clever policy-management is going to paper over the cracks. What's the point of 5-9's reliability, if it only applies 70% of the time?

Another overlooked part of QoS management is security. Can DDoS overload the packet-scheduling so that even the "platinum-class" apps won't get through? Does the QoS/policy infrastructure change or expand the attack surface? Do the compromises needed to match encryption + QoS introduce new vulnerabilities? Put simply, is it worth tolerating occasionally-glitchy applications, in order to reduce the risks of "existential failure" from outages or hacks? 

There are plenty of other "gotchas" about the idea of paid QoS, especially on mobile. I discussed them in a report last year (link) about "non-neutral" business models, where I forecast that this concept would have a very low revenue opportunity.

There's also another awkwardness: app developers generally don't care about network QoS enough to pay for more of it, especially at large-enough premiums to justify telcos' extra cost and pain of more infrastructure and IT (and lawyers). 

While devs might want to measure network throughput or latency, the general tendency is to work around the limitations, not pay to fix them. That's partly because the possibility isn't there today, but also because they don't want to negotiate with 1000 carriers around the world with different pricing schemes and tax/regulatory environments (not to mention the 300 million WiFi owners already mentioned). Most would also balk at paying for networks' perceived failings, or possibly to offset rent-seeking or questionable de-prioritisation. Startups probably don't have the money, anyway. 

Moreover - and to the core of this post - in most cases, it's better to use software techniques to "deal with" poor network quality, or avoid it. We already see a whole range of clever "adaptive" techniques employed, ranging from codecs that change their bit-rate and fidelity, through to forward error-correction, or pre-cacheing of data in advance if possible. A video call might drop back to voice-only, or even messaging as a fallback. Then there's a variety of ways of repairing damage, such as packet-loss concealment for VoIP. In some cases, the QoS-mitigation goes up to the UI layer of the app: "The person you're talking to has a poor connection - would you like to leave a voicemail instead?"

And this is where machine-learning and AI comes in. Because no matter how fast network technology is evolving - NFV & SDN, 5G, "network-slicing" or anything else - the world of software and cognitive intelligence is evolving faster still. 

I think that machine-learning and (eventually) AI will seriously damage the future prospects for monetising network QoS. As Martin points out regularly, you can't "put quality back into the network" once it's lost. But you can put quality, cognitive smarts or mitigation into the computation and app-logic at each end of the connection, and that's what already occurring and is about to accelerate further.

At the moment, most of the software mitigation techniques are static point solutions - codecs built-into the media engines, for instance. But the next generation is more dynamic. An early example is that of enterprise SD-WAN technology, which can combine multiple connections and make decisions about which application data to send down which path. It's mostly being used to combine cheap commodity Internet access connections, to reduce the need to spend much more on expensive managed MPLS WANs. In some cases, it's cheaper and more reliable to buy three independent Internet connections, mark and send the same packets down all of them simultaneously, and just use whichever arrives first at the other end to minimise latency. As I wrote recently (link), SD-WAN allows the creation of "Quasi-QoS".

Furthermore, an additional layer of intelligence and analytics allows the SD-WAN controller (sitting in the cloud) to learn which connections tend to be best, and under which conditions. The software can also learn how to predict warning-signs of problems and what the best fixes are. Potentially it could also signal to the app, to allow preventative measures to be taken - although this will obviously depend on the timescales involves (it won't be able to cope with millisecond transients, for instance).

But that is just the start, and is still just putting intelligence into the network, albeit an overlay.

What happens when the applications themselves get smarter? Many are already "network-aware" - they know if they're connected via WiFi or 4G, for example, and adapt their behaviour to optimise for cost, bandwidth or other variables. They may be instrumented to monitor quality and self-adapt, warn the user, or come up with mitigation strategies. They have access to location, motion-sensor and other APIs, that could inform them about which network path to choose.

But even that is still not really "learning" or AI. But now consider the next stage - perhaps a VoIP application spots glitches, but rather than an inelegant drop, it subtly adds an extra "um" or "err" in your voice (or just a beep) to buy itself an extra 200ms to wait for the network to catch up? Perhaps it is possible to send voice-recognised words and tone to a voice-regenerating engine at the far end, rather than the modulated wave-forms of your actual speech?

Or look forward another few years, and perhaps imagine that you have a "voice bot" that can take over the conversation on your behalf, within certain conversational or ethical guidelines. Actually, perhaps you could call it an "ambassador" - representing your views and empowered to take action in your absence if necessary. If two people in a trusted relationship can send their ambassadors to each others' phone, the computers can take over if there's a network problem. Your "mini-me" would be an app on your friend's or client's device and create "the illusion of realtime communications".
   
Obviously it would need training, trust and monitoring, but in some cases it might even generate better results. "Siri, please negotiate my mobile data plan renewal for the best price, using my voice". "Cortana, please ask this person out on a date, less awkwardly than I normally do" (OK, maybe not that one...)

Investment banks already use automated trading systems, so there are already examples of important decisions being made robotically. If the logic and computation can be extended locally to "the other end" - with appropriate security and record-keeping - then the need for strict network QoS might be reduced. 

Machine-learning may also be useful to mitigate risks from network unavailability, or security exploits. If the app knows from past experience that you're about to drive through a coverage blackspot, it can act accordingly in advance. The OS could suggest an alternative app or method for acheiving your underlying goal or outcome - whether that is communication or transaction - like a SatNav suggesting a new route when you miss a turn.



For some applications, maybe the network is only used a secondary approach, for error-correction or backup. In essence, it takes the idea of "edge computing" to its ultimate logical extension - the "edge" moves right out to the other user's device lor gateway, beyond the network entirely. (This isn't conceptually much different to a website's JavaScript apps running in your browser). 

Obviously, this approach isn't going to work ubiquitously. Network QoS will still be needed for transmitting unpredictable real-time data, or dealing with absolutely mission-critical applications. Heavy-lifting will still need to be done in the cloud - whether that's a Google search, or a realtime lookup in a sales database. Lightweight IoT devices won't support local computing and maintain low power consumption. But clever application design, plus cognitively-aware systems, can reduce the reliance on the access network for many cases. It could just be argued that this is just a lower quality threshold, but at a certain point that coincides with what is routinely available from a normal Internet connection, or perhaps two or three bonded or load-balanced together.

But overall, just as we expect to see robots taking over from humans in "automatable jobs", so too will we see computation and AI taking over from networks in dealing with "automatable data". The basis for the network "translocating" data becomes less of an issue, if the same data (or a first-approximation) can be generated locally to begin with.