Comcast & DPI - Caveat Inspector

I've argued for some time that many of the principles of ISPs' traffic engineering and packet inspection were flawed. That heavy-handed application of DPI would either result in what the medical industry calls 'false positives' (ie erroneous detection of something benign as malicious) or would lead to an escalating arms race by applications developers.

Although they still seem to be denying it, the web is a-splutter with accusations against US broadband provider Comcast. It's apparently doing something to block uploaded files - a process intended to counter things like filesharing, especially via BitTorrent.

Well, there's a great lesson to be learnt here about false positives. Apparently, the allegedly-non-existent traffic-shaping minions at Comcast forgot to think through what else might be affected. Like IBM Lotus Notes for example. Not to mention anyone using BitTorrent for legal purposes. Or assorted internally-developed corporate applications that might stop working.

Apparently there's now some sort of fix for this, I'd guess done behind the scenes between IBM and Comcast. While IBM will have the clout (and contacts) to do this, I'll bet that corporate developers won't.

I can also envisage some fun conversations....

"Hello, is that Mr Comcast Corporate Account Manager?..... yes, this is Mr Smith from XYZ Investment Bank. Well, we've got this application we developed... it uses a modified version of BitTorrent.... it distributes equity derivative pricing formulae to our traders.... it seems to have stopped working..... we've isolated the problem to your network. You started blocking it with no warning..... Yes, it's causing us some trouble...... Yes, quite a bit. About $300m so far...... well, if you could just hang on for a minute, maybe you'd like to speak to Mr SueEm-to-Bits our chief counsel? And Messrs Sarbanes & Oxley would like a word with you afterwards".

Meanwhile, I've noticed a distinct upswing in people talking recently about putting broadband and mobile traffic inside VPNs, or encrypting it.

My view is that a lot of the more onerous terms-of-service for broadband (fixed or wireless), or heavy-handed traffic shaping, are going to evaporate under commercial & PR pressure over the next year or so. And I'm expecting quite a lot of Comcast-type attempts to be outed by people monitoring traffic flows and reverse-engineering what's causing problems.

As I've said before - I can't get especially exercised about Net Neutrality. Because anyone stupid enough to block benign traffic (even if they don't like it for commercial reasons) is going to heavily burnt. I think the market will understand providers wanting to stop network integrity being threatened - but it will show zero tolerance for vindictive or accidental 'collateral damage'.


Edit

Just remembered a little snippet I heard last week. Apparently, P2P traffic has actually been dropping as a % of the total on the Internet over the last year or so. Can't remember the exact numbers, but it was something like 50% a year ago, but only 35% now. Not because it's being throttled...

....but because something else has grown faster: social networking. Blame MySpace, FaceBook, YouTube et al.

And while certain elements are easily trackable in DPI (facebook.com's IP address etc), the proliferating 1000's of plug-in are not. Are you going to traffic shape bits of Little Johnny in Iowa's MySpace page, but not others? Who are you going to bill when you decide that the Vibrating Hamster widget on my FaceBook page is consuming too much bandwidth?