Collaborative Puzzle Solving in Distributed Training

Will AI agents run all businesses? In episode 62 of Mixture of Experts, host Tim Hwang is joined by Gabe Goodhart, Kush Varshney and Marina Danilevsky to debrief Anthropic’s Project Vend. Next, do we still need massive data centers? We analyze DiLoCoX and discuss the possibility of distributed model training. Then, the New York Times released an article discussing how computer science education has changed in the era of AI; should people still study computer science? Finally, is the paper review process broken? And is it AI’s fault? All that and more on today’s episode of Mixture of Experts. 

00:00 – Intro 

01:07 -- Anthropic’s Project Vend 

13:38 -- DiLoCoX 

25:56 -- Computer science education 

40:57 -- AI prompts in papers 

The opinions expressed in this podcast are solely those of the participants and do not necessarily reflect the views of IBM or any other organization or entity.