Vanishing Gradients Episode 58: Building GenAI Systems That Make Business Decisions with Thomas Wiecki (PyMC Labs)
Sep 9, 2025
Thomas Wiecki, founder of PyMC Labs and co-author of PyMC, dives into how generative AI can shape business decisions. He discusses using large language models as synthetic consumers to test product ideas, revealing the efficiency of AI over traditional surveys. Thomas emphasizes Bayesian modeling's role in providing trustworthy insights and navigating complex data. His experience with Colgate highlights the iterative design of AI systems for better product and marketing strategies, urging a balance between innovative models and reliability.
AI Snips
Chapters
Transcript
Episode notes
Bayes + LLMs For Real Decisions
- Bayesian causal models plus LLM-based "synthetic consumers" can simulate realistic human survey responses.
- That combo provides predictions, counterfactuals, and richer decision-making than simple retrieval-style LLM apps.
Colgate Synthetic Survey Case Study
- PyMC Labs worked with Colgate to test many novel toothpaste ideas using LLM-based synthetic surveys.
- The synthetic survey averages correlated about 90% with human survey results on holdout data.
Reasoning + Embeddings Beats Direct Likert
- Having LLMs generate reasoning (free-text) then mapping to Likert embeddings improved fidelity.
- Prompting demographic context into the prompt let models emulate subgroup behavior without fine-tuning.