Episode 30: Lessons from a Year of Building with LLMs (Part 2)

Hugo speaks about Lessons Learned from a Year of Building with LLMs with Eugene Yan from Amazon, Bryan Bischof from Hex, Charles Frye from Modal, Hamel Husain from Parlance Labs, and Shreya Shankar from UC Berkeley.

These five guests, along with Jason Liu who couldn't join us, have spent the past year building real-world applications with Large Language Models (LLMs). They've distilled their experiences into a report of 42 lessons across operational, strategic, and tactical dimensions, and they're here to share their insights.

We’ve split this roundtable into 2 episodes and, in this second episode, we'll explore:

• An inside look at building end-to-end systems with LLMs;
• The experimentation mindset: Why it's the key to successful AI products;
• Building trust in AI: Strategies for getting stakeholders on board;
• The art of data examination: Why looking at your data is more crucial than ever;
• Evaluation strategies that separate the pros from the amateurs.

Although we're focusing on LLMs, many of these insights apply broadly to data science, machine learning, and product development, more generally.

LINKS

• The livestream on YouTube
• The Report: What We’ve Learned From A Year of Building with LLMs
• About the Guests/Authors <-- connect with them all on LinkedIn, follow them on Twitter, subscribe to their newsletters! (Seriously, though, the amount of collective wisdom here is 🤑
• Your AI product needs evals by Hamel Husain
• Prompting Fundamentals and How to Apply them Effectively by Eugene Yan
• Fuck You, Show Me The Prompt by Hamel Husain
• Vanishing Gradients on YouTube
• Vanishing Gradients on Twitter
• Vanishing Gradients on Lu.ma