In episode 22 of Recsperts, we welcome Prabhat Agarwal, Senior ML Engineer, and Aayush Mudgal, Staff ML Engineer, both from Pinterest, to the show. Prabhat works on recommendations and search systems at Pinterest, leading representation learning efforts. Aayush is responsible for ads ranking and privacy-aware conversion modeling. We discuss user and content modeling, short- vs. long-term objectives, evaluation as well as multi-task learning and touch on counterfactual evaluation as well.

In our interview, Prabhat guides us through the journey of continuous improvements of Pinterest's Homefeed personalization starting with techniques such as gradient boosting over two-tower models to DCN and transformers. We discuss how to capture users' short- and long-term preferences through multiple embeddings and the role of candidate generators for content diversification. Prabhat shares some details about position debiasing and the challenges to facilitate exploration.
With Aayush we get the chance to dive into the specifics of ads ranking at Pinterest and he helps us to better understand how multifaceted ads can be. We learn more about the pain of having too many models and the Pinterest's efforts to consolidate the model landscape to improve infrastructural costs, maintainability, and efficiency. Aayush also shares some insights about exploration and corresponding randomization in the context of ads and how user behavior is very different between different kinds of ads.
Both guests highlight the role of counterfactual evaluation and its impact for faster experimentation.

Towards the end of the episode, we also touch a bit on learnings from last year's RecSys challenge.

Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.
Don't forget to follow the podcast and please leave a review

• (00:00) - Introduction
• (03:51) - Guest Introductions
• (09:57) - Pinterest Introduction
• (21:57) - Homefeed Personalization
• (47:27) - Ads Ranking
• (01:14:58) - RecSys Challenge 2023
• (01:20:26) - Closing Remarks

• Prabhat Agarwal on LinkedIn
• Aayush Mudgal on LinkedIn
• RecSys Challenge 2023
• Pinterest Engineering Blog
• Pinterest Labs
• Prabhat's Talk at GTC 2022: Evolution of web-scale engagement modeling at Pinterest
• Blogpost: How we use AutoML, Multi-task learning and Multi-tower models for Pinterest Ads
• Blogpost: Pinterest Home Feed Unified Lightweight Scoring: A Two-tower Approach
• Blogpost: Experiment without the wait: Speeding up the iteration cycle with Offline Replay Experimentation
• Blogpost: MLEnv: Standardizing ML at Pinterest Under One ML Engine to Accelerate Innovation
• Blogpost: Handling Online-Offline Discrepancy in Pinterest Ads Ranking System

Papers:

• Eksombatchai et al. (2018): Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time
• Ying et al. (2018): Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Pal et al. (2020): PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest
• Pancha et al. (2022): PinnerFormer: Sequence Modeling for User Representation at Pinterest
• Zhao et al. (2019): Recommending what video to watch next: a multitask ranking system

General Links:

• Follow me on LinkedIn
• Follow me on X
• Send me your comments, questions and suggestions to marcel.kurovski@gmail.com
• Recsperts Website