This episode is a perfect fit for anyone who wants to learn how to apply AI into their product especially for high fidelity analytics use cases.
Utkarsh Ohm is Director of AI at Thoughtspot. He previously was a Co-Founder & eng leader at various startups. In this episode, we cover applying LLMs alongside other models to customer facing products. We discuss where LLMs in particular are helpful vs traditional ML models, why text to SQL use cases need to be supplemented by internal insights to improve accuracy, and how to approach data security for LLMs. We deep dive specifically into Sage, Thoughtspot’s LLM enabled product and the infrastructure that allows Sage to produce accurate insights for customers. Over the course of the episode, we discuss plenty of advice for teams building with LLMs to learn from and take into account.
Where to Find Utkarsh:
* Twitter: https://twitter.com/utkarshohm
* LinkedIn: https://www.linkedin.com/in/utkarshohm/
Where to Find Shomik:
* Twitter: https://twitter.com/shomikghosh21
* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/
* Newsletter: https://www.shomik.substack.com
* Podcast: Apple Podcasts, Spotify, YouTube.
In this episode, we cover:
(01:35) - Utkarsh’s Journey to Becoming Director of AI at Thoughtspot
(03:14) - Learning AI/ML on the Job
(05:08) - Emergent Capabilities of LLMs Leading to Experimentation
(06:52) - AI Used in Thoughtspot Products in 2018
(10:30) - The ChatGPT Moment
(12:16) - How Sage (the LLM Powered Product) Works
(13:44) - 3 Key Capabilities of Building an Analytical Q&A System
(16:15) - Why Thoughtspot Doesn’t Use LLMs for Text to SQL
(20:35) - Why Models Currently Aren’t Good Enough for Analytical Understanding
(23:15) - The Infra Architecture Behind Sage
(29:55) - How to Approach Data Security for Applied AI Products
(33:15) - Customer Outcomes from Using Sage
(38:30) - Managing Hallucinations in the Hands of Customers
(42:40) - Advice for Teams Building LLM-enabled Products
Show Notes:
* Security and Risk Mitigation in an LLM World
* ThoughtSpot Sage Product
How to Subscribe:
Available on Apple Podcasts, Spotify, YouTube.
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit
shomik.substack.com
Software Snack Bites