Exploring the Development of Open-Source LLM Project and Future Plans

Artificial General Intelligence, RLHF’s application in AI, and how entrepreneurs can enter the AI industry: Meta’s AI Research Scientist Thomas Scialom gives us behind-the-scenes insights into developing Llama 2 and what’s in the works for Llama 3. With host Jon Krohn, he discusses the future of Artificial General Intelligence, why the Galactica science-focused LLM was taken down, and what he learned from it.

This episode is brought to you by AWS Inferentia, by Grafbase, the unified data layer, and by Modelbit, for deploying models in seconds. Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.

In this episode you will learn:
• Llama 2: Behind the Scenes of Today’s Top Open-Source LLM [05:04]
• Responsible use of Llama 2 [15:26]
• Toolformer: LLM That Learns How to Use External Tools [24:57]
• Galactica: The Science-Specific LLM and Why It Was Brought Down [36:57]
• Is AGI Around the Corner? [57:03]
• Advice for AI entrepreneurs [1:05:46]
• How Thomas develops and manages large-scale AI projects [1:14:42]

Additional materials: www.superdatascience.com/713