Software Engineering Radio - the podcast for professional software developers SE Radio 698: Srujana Merugu on How to build an LLM App
Dec 9, 2025
Srujana Merugu, an AI researcher with extensive experience at tech giants like Google and Amazon, dives into the intricacies of building LLM-based applications. She clarifies concepts such as generative vs. predictive AI and explains the importance of transformer architecture. The discussion covers practical use cases and where LLMs might not be optimal. Srujana also shares insights on model selection, safety guardrails, and emerging trends like multi-sensory AI. Plus, she offers tips on staying current in the fast-evolving AI landscape.
AI Snips
Chapters
Transcript
Episode notes
Generative vs Predictive AI Clarified
- Generative AI produces complex artifacts like text, images, audio and differs from predictive models that output labels or scores.
- Generative models excel at creation while predictive (discriminative) models remain better for high-accuracy labeling tasks.
Use Pre-Train, Fine-Tune, Then Align
- Pre-train models on massive unlabeled data to build broad knowledge, then fine-tune on labeled data for specific tasks.
- Apply alignment (e.g., RLHF) to make outputs helpful, safe, and aligned with human preferences.
Why Transformers Scaled LLMs
- The transformer architecture uses attention to process tokens in parallel and enabled scaling to massive language models.
- Encoders produce contextual embeddings while decoders generate text token-by-token; many LLMs use decoder-only designs.