Cables2Clouds

What Does an AI Data Center Look Like? - C2C046

Nov 13, 2024

38:14

World-renowned network expert Peter Jones returns to the Cables to Clouds podcast, sharing his unparalleled insights into the electrifying evolution of AI networking. We promise listeners a captivating journey through the latest advancements and investments reshaping data centers to meet the high demands of AI. Explore with us the nuanced differences between AI for networking and networking for AI, as we shed light on how these developments are capturing Wall Street’s attention and web scalers’ ambitions.

Venture into the intricate world of networking technology as we tackle the tough challenges of packet reliability and error management, pondering whether the link layer should shoulder more of TCP's load. Discover the strategies of tech behemoths like Google, Meta, and Microsoft as they navigate the complex terrain of cost, power, and reliability. From innovative solutions like liquid cooling and disaggregated power to the futuristic promise of nuclear energy, we'll dive deep into the technological shifts required to power tomorrow's networks.

Finally, Peter lends his expertise to unravel the scaling challenges of GPU networks, especially poignant in the realms of Bitcoin mining and cloud computing. Listen as we draw parallels between overclocking and the immense strain on GPUs in mining farms, all while questioning the sustainability of these massive endeavors. As we explore Microsoft's strategic energy maneuvers and the potential of disaggregated systems, this episode promises a comprehensive look at the future of data centers and the efficient management of AI workloads. Join us for an enlightening conversation that paves the way for understanding the dynamic future of networking.

Links:
https://www.semianalysis.com/

Google
https://cloud.google.com/blog/topics/systems/the-evolution-of-googles-jupiter-data-center-network

Meta
https://engineering.fb.com/2024/08/05/data-center-engineering/roce-network-distributed-ai-training-at-scale/
https://engineering.fb.com/2023/11/15/networking-traffic/watch-metas-engineers-on-building-network-infrastructure-for-ai/

Microsoft
https://news.microsoft.com/source/features/ai/in-house-chips-silicon-to-service-to-meet-ai-demand/
https://azure.microsoft.com/en-us/blog