Balancing Real-Time Recommendations and System Scalability

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Sanket works as a Senior Machine Learning Engineer at Spotify working on building end-to-end audio recommender systems. Models built by his team are used across Spotify in many different products including Discover Weekly and Autoplay. MLOps podcast #232 with Sanket Gupta, Senior Machine Learning Engineer at Spotify // RecSys at Spotify. A big thank you to LatticeFlow for sponsoring this episode! LatticeFlow - https://latticeflow.ai/ // Abstract LLMs with foundational embeddings have changed the way we approach AI today. Instead of re-training models from scratch end-to-end, we instead rely on fine-tuning existing foundation models to perform transfer learning. Is there a similar approach we can take with recommender systems? In this episode, we can talk about: a) how Spotify builds and maintains large-scale recommender systems, b) how foundational user and item embeddings can enable transfer learning across multiple products, c) how we evaluate this system d) MLOps challenges with these systems // Bio Sanket works as a Senior Machine Learning Engineer on a team at Spotify building production-grade recommender systems. Models built by my team are being used in Autoplay, Daily Mix, Discover Weekly, etc. Currently, my passion is how to build systems to understand user taste - how do we balance long-term and short-term understanding of users to enable a great personalized experience. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://sanketgupta.substack.com/ Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584 Sanket's blogs on Medium in the past: https://medium.com/@sanket107 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sanket on LinkedIn: www.linkedin.com/in/sanketgupta107 Timestamps: [00:00] Sanket's preferred coffee [00:37] Takeaways [02:30] RecSys are RAGs [06:22] Evaluating RecSys parallel to RAGs [07:13] Music RecSys Optimization [09:46] Dealing with cold start problems [12:18] Quantity of models in the recommender systems [13:09] Radio models [16:24] Evaluation system [20:25] Infrastructure support [21:25] Transfer learning [23:53] Vector database features [25:31] Listening History Balance [26:35 - 28:06] LatticeFlow Ad [28:07] The beauty of embeddings [30:13] Shift to real-time recommendation [34:05] Vector Database Architecture Options [35:30] Embeddings drive personalized [40:16] Feature Stores vs Vector Databases [42:33] Spotify product integration strategy [45:38] Staying up to date with new features [47:53] Speed vs Relevance metrics [49:40] Wrap up