In this episode of ACM ByteCast, Bruke Kifle hosts 2024 ACM Prize in Computing recipient Torsten Hoefler, a Professor of Computer Science at ETH Zurich (the Swiss Federal Institute of Technology), where he serves as Director of the Scalable Parallel Computing Laboratory. He is also the Chief Architect for AI and Machine Learning at the Swiss National Supercomputing Centre (CSCS). His honors include the Max Planck-Humboldt Medal, an award for outstanding mid-career scientists; the IEEE CS Sidney Fernbach Award, which recognizes outstanding contributions in the application of high-performance computers; and the ACM Gordon Bell Prize, which recognizes outstanding achievement in high-performance computing. He is a member of the European Academy of Sciences (Academia Europaea), a Fellow of IEEE, and a Fellow of ACM.
In the interview, Torsten reminisces on early interest with multiple computers to solve problems faster and on building large cluster systems in graduate school that were later turned into supercomputers. He also delves into high-performance computing (HPC) and its central role in simulation and modeling across all modern sciences. Bruke and Torsten cover the various requirements that power HPC, the intersection of HPC and recent innovations in AI, and his key contributions in popularizing 3D parallelism for training AI models. Torsten highlights challenges, such as AI’s propensity to cheat, as well as the promise of turning reasoning models into scientific collaborators. He also offers advice to young researchers on balancing academic learning with industry exposure.