Last monday Martin Geddes and PNSOL organized a workshop (Future of Broadband Flyer) on their vision of the future direction of “multiplexed packet switched networks” (aka broadband and/or Internet). Contention and discussion garantueed, as Martin and Neil c.s. state that we are on a track that leads to doom: kind of like the first climate change prophets in a room full of petrolheads 
.
The lively discussion sharpened my thoughts and views: I do not agree with everything that was claimed, yet the issue at hand is worth to be investigated.
Statistical multiplexing of information packets is a fascinating subject: the theory is complex, and the practice as embodied in the Internet is a revolution in society as big as any in history. As Dean Bubley said, the value generated for society by the Internet in only a few decades as hard to overestimate, our children cannot imagine a world without it, so we should cherish what we have and be very careful in applying “improvements”. This warrants both an open mind to any proposal to improve and a healthy respect for what has been achieved by many great minds who kept on researching and tinkering to get what we currently have. After all, the Porsche 911 sounds like a bad design with lousy handling on paper, but look what 40 years of improvements got us…
The core insight that has got my attention ( long before the workshop) is that statistical multiplexing of information packets has made a global information network like the Internet possible and affordable: yet the price to pay is that “noise” or imperfections are added as the combined load rises. And once “noise” (delay, jitter, loss of packets) is added you cannot reverse the degradation, its a one way street. In many cases the noise is inconsequential, sometimes it is not.
The best analogy ( damn analogies…yet we cannot communicate without them) is a highway with cars. As traffic density increases first nothing significant happens, but after a certain level of traffic density is reached the average speed is reduced (delay) and the variation in arrival time increases (jitter). The chance of an accident increases (loss). Once the traffic density reaches “the cliff” (maximum level) any minor perturbance causes a collapse of the flow, reducing the throughput to almost nothing ( aka “traffic jam”). A well known effect on highways.
The analogy fails in many aspects: if only where on a multiplexed packet switched network the changes in offered load can vary instantaneously and very fast, and loss of packets is acceptable, even is used on purpose as a signaling mechanism. On the Internet we have a flow control protocol like TCP that is designed to reduce the sending speed when somewhere in the path the flow level reaches “the cliff”. When TCP senses that packets do not arrive at the destination (loss as signaling) it backs off, only to try again later if the speed can be raised again.
The typical reaction time of this control loop is dependent on the round trip time of packets sometimes elongated by a network design flaw called “buffer bloat“. Any traffic phenomenon like changes in offered load that is faster than the control loop of TCP can react to will not be compensated for: it even might have adverse effects as multiple effects including the control loop of TCP interact with each other.
The claim Martin c.s. make is that as accces speeds increase (FttH, HFC networks) the volatility of variations in a multiplexed link starts to outpace the TCP control loop, leading to more and more transient “traffic jams” and even collapse. Which could be true: they showed some measurements of the variation in delay of packets in real life that indicated that there might be a problem. (Much more data needed however).
They use the graph below (click to expand so you can see the full graph) to make their point. The amount of time (delay) it takes to send a packet of information from sender to receiver depends on:
- distance (speed of EM waves is finite)
- number of routers which convert light to electrical to light
- serialization delay (you have to wait until the last bit is there)
- transient delays (contention in buffers, loss and resending etc.), also called non-stationarity
Without the transients TCP can do a great job.
So far so good: I would like to see much more measurements and analysis of data to determine if these transients are a) a new phenomenon b) increasing in number and size c) are the cause of big problems d) caused by what we think is the source. Worth the effort,
Assuming that the transients indeed prove to be a serious problem the question arises what the remedy should be.
(PNSOL proposes (not part of the workshop) that the network operator intervenes at the ingress-point of a section of a network. The intervention is based on the value of a certain type of stream of packets, and the sensitivity for that stream for loss and delay/jitter. For instance: VOIP is sensitive to delay and jitter, not so much to loss. Mail is quite insensitive to delay and jitter, so you prioritize VOIP and delay mail. The intervention makes sure that at no point downstream contention arises, so all loss and delay are distributed at the ingress point. I guess this can work as advertised, but….)
The organizers of the workshop went on with statements that I question:
- because of the transient non-stationarity effects we need a new flow control paradigm to be able to utilize the resources (capacity) much better, adding bandwidth is not a solution
- operators have an unsustainable (or rather very risky) business model if they don’t apply the new paradigm, because they will be taken by surprise when transients lead to collapse or go broke on adding bandwidth
- networks are to become trading places for “noise/imperfections”
All that I have learned over the years is that bandwidth is cheap and running a network below maximum utilization will keep the transients low. (Again, if the fast transients are indeed a problem we need extra measures to remedy that, not necessarily the way PNSOL envisions that).
The business models of fixed line operators are not very dependent on the cost of amount of bandwidth offered (both transit and backhaul or access), provided the physical infrastructure is good enough (aka fiber). Yes there is a problem if you run over copper or underinvest. The operators balk at the one-time investments needed to go to fiber, as no CEO wants to send a message to shareholders that the rich dividends will be absent for a decade or so. Wireless is a totally different situation: networks taken by surprise at the demand and the type of demand (signaling load), claiming spectrum shortage as an competitive strategy to keep ouy contenders, shift of revenue from voice and text to lower revenues of broadband while investments are required create a fuzzy picture of what reality is.
Aiming for 100 % utilization so you can delay investments, at the price of the operator deciding what valuable is and what not creates a big moral hazard and a potential destruction of future innovations. You immediately create an incentive NOT to invest in capacity and create artifical scarcity which you as an operator can monetize. The operator gets to decide what is valuable and what is not. A bad deal for society.
As I have argued before the interpretation of transmitted (or even not-transmitted) data is already dependent on the particular sender and receiver, the value of the interpretation is even more specific. So no operator should interfere. Martin proposes that the network becomes a trading place for end-users, trading loss and delay options which the operator only executes. Even if that would be possible (information asymmetry, no options to leave the market, very hard to make informed decisions all the time for normal people) it is a complex and costly solution for a problem created by scarcity, by lack of investment in infrastructure.
Having said that, our regulators and politicians are at loss to get the investments in new infrastructure going. I have yet to see regulation that really incentivizes investments. The Network Neutrality debate is about the same issues as discussed here. There is is complexe emergent relationship between network design, network operations (capacity and management), revenue for the network operator and value as experienced by users, wether we like it or not. And it exists today.
Workshops like these help to develop our conceptual framework on how to deal with these issues.