Customizing Large Language Models in Perplexity and Importance of Low Latency in AI Systems

10min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

The chapter highlights customizing the large pre-trained language models in Perplexity with models like GPT-40 and LAMA-3, emphasizing speed advantages over cloud models and ongoing efforts to enhance performance for complex queries. It discusses the importance of being model agnostic in AI product development to provide optimal answers regardless of the model used, focusing on low latency inspired by Google's tail latency concept. Additionally, it explores the complexities of managing latency and scaling compute capacity in startups, including considerations for in-house compute versus cloud solutions and the competition among cloud service providers like AWS, Google Cloud, and Azure.

Arvind Srinivas is CEO of Perplexity, a company that aims to revolutionize how we humans find answers to questions on the Internet. Please support this podcast by checking out our sponsors:
– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off
– ShipStation: https://shipstation.com/lex and use code LEX to get 60-day free trial
– NetSuite: http://netsuite.com/lex to get free product tour
– LMNT: https://drinkLMNT.com/lex to get free sample pack
– Shopify: https://shopify.com/lex to get $1 per month trial
– BetterHelp: https://betterhelp.com/lex to get 10% off

Transcript: https://lexfridman.com/aravind-srinivas-transcript

EPISODE LINKS:
Aravind’s X: https://x.com/AravSrinivas
Perplexity: https://perplexity.ai/
Perplexity’s X: https://x.com/perplexity_ai

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips

SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman

OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(10:52) – How Perplexity works
(18:48) – How Google works
(41:16) – Larry Page and Sergey Brin
(55:50) – Jeff Bezos
(59:18) – Elon Musk
(1:01:36) – Jensen Huang
(1:04:53) – Mark Zuckerberg
(1:06:21) – Yann LeCun
(1:13:07) – Breakthroughs in AI
(1:29:05) – Curiosity
(1:35:22) – $1 trillion dollar question
(1:50:13) – Perplexity origin story
(2:05:25) – RAG
(2:27:43) – 1 million H100 GPUs
(2:30:15) – Advice for startups
(2:42:52) – Future of search
(3:00:29) – Future of AI