Joining the discussion is Hamel Husain, a seasoned ML engineer and open-source contributor, who shares invaluable insights on debugging generative AI systems. He emphasizes that understanding data is key to fixing broken AI applications. Hamel advocates for spreadsheet error analysis over complex dashboards. He also highlights the pitfalls of trusting LLM judges blindly and critiques existing AI dashboard metrics. His practical methods will transform how developers approach model performance and iteration in AI.

Tyler Dunn, CEO and co-founder of Continue, discusses the transformative role of open-source AI coding assistants. He delves into the crucial balance between developer control and AI capabilities, highlighting how customization can empower software engineers. The conversation covers the evolution from autocomplete to intelligent code suggestions and the future of fine-tuning AI models on personalized data. Dunn emphasizes the importance of integration, tailored experiences, and maintaining trust as developers navigate the ever-evolving landscape of AI in coding.

Hugo chats with Alex Strick van Linschoten, a Machine Learning Engineer at ZenML, who has documented over 400 real-world LLM deployments. They discuss the challenges in deploying AI agents, like hallucinations and cascading failures. Alex reveals practical lessons from corporate giants like Anthropic and Klarna, focusing on structured workflows that enhance reliability. He highlights the evolution of LLM capabilities and shares case studies that underscore the importance of prompt engineering and effective error handling in building robust AI systems.

In this discussion, Alex Andorra, host of the Learning Bayesian Statistics podcast and an expert in Bayesian stats and sports analytics, joins Hugo to explore the intersection of AI, education, and product development. They reveal how Bayesian thinking aids in overcoming challenges in AI applications and the critical importance of iteration and first principles. The conversation also highlights the influence of commercial interests on experimentation, the evolution of teaching methods in tech, and the intricate world of deploying AI with LLMs.

Ben Taylor, CEO of VEOX Inc., Joe Reis, co-founder of Ternary Data, and Juan Sequeda, Principal Scientist at Data.World, discuss the evolution of AI from prompt engineering to goal engineering. They explore whether generative AI is more akin to an electrifying revolution or a blockchain phase. The panel highlights the importance of tackling the POC-to-production gap, understanding AI's failure modes, and balancing executive enthusiasm with employee workload. They also examine how AI's combinatorial abilities can redefine strategies, paralleling the success of AlphaZero in gaming.

In this conversation with Charles Frye, Developer Advocate at Modal, listeners gain insights into the intricate world of GPUs and their critical role in AI and LLM development. Charles explains the importance of VRAM and how memory can become a bottleneck. They tackle practical strategies for optimizing GPU usage, from fine-tuning to training large models. The discussion also highlights a GPU Glossary that simplifies complex concepts for developers, along with insights on quantization and the economic considerations in using modern hardware for efficient AI workflows.

Hugo chats with Ravin Kumar, a Senior Research Data Scientist at Google Labs, whose career journey includes roles at SpaceX and Sweetgreen. They delve into the balance between technical rigor and practical utility in generative AI. Ravin shares insights on building scalable AI systems, such as using Gemma to optimize bakery operations. He emphasizes the critical role of UX in AI adoption, showcases the Notebook LM tool in action, and explores how AI can aid small businesses—demonstrating the transformative power of accessible technology.

Jason Liu, an independent AI consultant with a background at Meta and Stitch Fix, joins the discussion. He shares insights into structuring valuable consulting contracts and shifting from hourly billing to larger deals. Engaging in a live role-play, Jason coaches the host on effective client interaction and pricing strategies. The conversation also highlights the shift from deterministic to probabilistic AI systems, emphasizing the importance of understanding client motivations and fostering meaningful relationships in the evolving freelance landscape.

Join Sander Schulhoff, a specialist in prompt engineering, Philip Resnik, a computational linguistics professor, and Dennis Peskoff from Princeton as they delve into the cutting-edge world of AI. They explore the security risks of prompt hacking and its implications for military use. Discussion highlights include the evolving role of generative AI across various fields, innovative techniques for improving AI self-criticism, and the pressing need for energy-efficient large language models. Their insights offer a fascinating glimpse into the future of AI research.

Hugo speaks with three leading figures from the world of AI research: Sander Schulhoff, a recent University of Maryland graduate and lead contributor to the Learn Prompting initiative; Philip Resnik, professor at the University of Maryland, known for his pioneering work in computational linguistics; and Dennis Peskoff, a researcher from Princeton specializing in prompt engineering and its applications in the social sciences. This is Part 1 of a special two-part episode, prompted—no pun intended—by these guys being part of a team, led by Sander, that wrote a 76-page survey analyzing prompting techniques, agents, and generative AI. The survey included contributors from OpenAI, Microsoft, the University of Maryland, Princeton, and more. In this first part, we’ll explore the critical role of prompt engineering, & diving into adversarial techniques like prompt hacking and the challenges of evaluating these techniques. we’ll examine the impact of few-shot learning and the groundbreaking taxonomy of prompting techniques from the Prompt Report. Along the way, we’ll uncover the rich history of natural language processing (NLP) and AI, showing how modern prompting techniques evolved from early rule-based systems and statistical methods. we’ll also hear how Sander’s experimentation with GPT-3 for diplomatic tasks led him to develop Learn Prompting, and how Dennis highlights the accessibility of AI through prompting, which allows non-technical users to interact with AI without needing to code. Finally, we’ll explore the future of multimodal AI, where LLMs interact with images, code, and even music creation. Make sure to tune in to Part 2, where we dive deeper into security risks, prompt hacking, and more. LINKS The livestream on YouTube The Prompt Report: A Systematic Survey of Prompting Techniques Learn Prompting: Your Guide to Communicating with AI Vanishing Gradients on Twitter Hugo on Twitter Vanishing Gradients' lu.ma calendar Vanishing Gradients on YouTube