AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
From Aerospace to AI: A Journey Through Search and Prompt Engineering
This chapter chronicles the speaker's transition from aerospace engineering to search technology and prompt engineering in machine learning. They share insights on their unexpected journey, emphasizing how their background seamlessly aligned with their new role in the co-pilot team.
John Berryman moved from aerospace engineering to search, then to ML and LLMs. His path: Eventbrite search → GitHub code search → data science → GitHub Copilot. He was drawn to more math and ML throughout his career.
RAG Explained
"RAG is not a thing. RAG is two things." It breaks into:
These should be treated as separate problems to optimize.
The Little Red Riding Hood Principle
When prompting LLMs, stay on the path of what models have seen in training. Use formats, structures, and patterns they recognize from their training data:
Models respond better to familiar structures.
Testing Prompts
Testing strategies:
Managing Token Limits
When designing prompts, divide content into:
Prioritize content by:
Even with larger context windows, efficiency remains important for cost and latency.
Completion vs. Chat Models
Chat models are winning despite initial concerns about their constraints:
Applications: Workflows vs. Assistants
Two main LLM application patterns:
Breaking Down Complex Problems
Two approaches:
Example: For SOX compliance, break horizontally (understand control, find evidence, extract data, compile report) and vertically (different audit types).
On Agents
Agents exist on a spectrum from assistants to workflows, characterized by:
Best Practices
For building with LLMs:
John Berryman:
Nicolay Gerold:
00:00 Introduction to RAG: Retrieval and Generation
00:19 Optimizing Retrieval Systems
01:11 Introducing John Berryman
02:31 John's Journey from Search to Prompt Engineering
04:05 Understanding RAG: Search and Prompt Engineering
05:39 The Little Red Riding Hood Principle in Prompt Engineering
14:14 Balancing Static and Dynamic Elements in Prompts
25:52 Assistants vs. Workflows: Choosing the Right Approach
30:15 Defining Agency in AI
30:35 Spectrum of Assistance and Workflows
34:35 Breaking Down Problems Horizontally and Vertically
37:57 SOX Compliance Case Study
40:56 Integrating LLMs into Existing Applications
44:37 Favorite Tools and Missing Features
46:37 Exploring Niche Technologies in AI
52:52 Key Takeaways and Future Directions
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode