In this dynamic discussion, Jonathan Frankle, Chief Scientist at MosaicML, shares insights on their $1.3 billion acquisition by Databricks. Lin Qiao, CEO of Fireworks AI, talks about optimizing PyTorch for inference. Aman Sanger from Cursor reveals innovative memory strategies for AI coding. Aravind Srinivas discusses the impressive growth of Perplexity AI, hitting 1 million installs, while Jeremy Howard emphasizes the need for accessible AI. Together, they explore the vibrant AI startup landscape showcased at NeurIPS 2023, reflecting on innovation, collaboration, and the future of technology.

In this discussion, Joel Hestness, a principal research scientist and lead of the core machine learning team at Cerebras, dives into the groundbreaking Wafer Scale Engine 3. He explains how this custom silicon surpasses traditional AI hardware, focusing on its unique architecture and memory capabilities. Joel also covers advancements in large language model training, innovative optimization techniques, and the integration of open-source frameworks like PyTorch. Additionally, he shares exciting research on weight-sparse training and novel optimizers that leverage higher-order statistics.