PING Faster Network design with simpler hardware: TCP Flow control and ECN.
In this episode of PING, APNIC Chief Scientist Geoff Huston shares a story from the recent AusNOG in Melbourne and connects it to measurement work at APNIC Labs, exploring how modern IP flow control manages ‘fair shares’ of the network.
At AusNOG 2025, Geoff attended a talk by Lincoln Dale of Amazon AWS titled “No Packet Left Behind: AWS’s Approach to Building and Operating Reliable Networks”. The presentation examined how AWS scales its data centre networks, highlighting massive investments in high-speed routers and switches to support both global internet services and the vast flows of traffic between servers and other Amazon resources.
What AWS doesn’t do is rely on highly complex protocols like Segment Routing over IPv6 (SRv6), Resource Reservation Protocol (RSVP), or other modern traffic engineering techniques unless absolutely necessary. Instead, they use a radically simplified, on-chip model of data management, pushing as much processing as possible into a single VLSI circuit and minimizing the amount of ‘smart’ work in the network. The question is: How can simplifying the IP stack to this extent actually work?
Geoff has long been sceptical of higher-layer protocols that try to manage bandwidth reservation and shaping. He recalls an earlier attempt by Digital Equipment Corporation (DEC) to signal congestion with Explicit Congestion Notification (ECN), a mechanism that still exists in the protocol stack and now underpins new bandwidth management approaches such as Apple and Comcast’s ‘L4S’.
APNIC Labs has measured how the wider Internet responds to ECN signals using an advertising-based model, and the results suggest this approach struggles outside tightly controlled, ‘walled garden’ networks. He contrasts this with advances in flow control through Google’s BBR, now in its third version, which refines the aggressive, bandwidth-seeking behaviour of TCP window management.
Read more about the story of IP, flow control and the modern Internet on the APNIC Blog, and the AusNOG website (video recordings of Lincon Dale’s talk and others should be released shortly)
- Measuring Explicit Congestion Notification (ECN) (Geoff Huston, APNIC Blog)
- Notes from AusNOG 2025 (Geoff Huston, APNIC Blog)
- The AusNOG 2025 program (AusNOG Website, videos to be released shortly)