Causal AI in Personalization | Dima Goldenberg Ep 19 | CausalBanditsPodcast.com
Jul 1, 2024
auto_awesome
Dima Goldenberg from Booking.com discusses building causal recommender systems, radical experimentation culture, and the importance of operations research classes. Topics include human psychology insights, optimizing promotions, dealing with noise in data, and childhood game lessons in work strategies.
Using causal inference in recommender systems goes beyond predictive accuracy by focusing on causal impact.
Aligning metrics and data collection processes between online and offline evaluations is crucial for evaluating models.
Connecting technical work with business impact is essential for deploying machine learning solutions effectively.
Understanding human behavior and causality requires humility, continuous testing, and counterfactual thinking.
Deep dives
Integrating Causal Inference in Recommendations
In the podcast, the guest discusses how personalizing recommendations on commerce platforms involves more than just predictive accuracy. By merging the concept of recommender systems with causal inference, the team at booking.com aims to move beyond correlation-based methodologies. Through A/B testing at scale, they explore not just better accuracy, but also the causal impact of recommendations, focusing on incremental changes that impact customer behavior.
Strategies for Online Model Evaluation
The podcast delves into the challenges of evaluating models online, emphasizing the importance of aligning metrics and data collection processes between online and offline evaluations. Continuous experimentation, portfolio approaches to modeling, and simplicity in models are highlighted as key strategies. The team aims to maintain consistency across different platforms and stay prepared for dynamic changes in customer behavior.
Navigating Business Stakeholder Communication
In navigating communication with business stakeholders, the guest stresses the significance of connecting technical work with business impact. Understanding use cases, business needs, and customer benefits is crucial in deploying machine learning solutions effectively. By aligning technical efforts with desired business outcomes and continuously monitoring and adapting based on feedback, the team at booking.com strives to optimize customer experience.
Challenges of Causal Inference and Human Psychology
The podcast explores the challenges in grasping causality and human psychology, highlighting the unpredictable and inconsistent nature of human behavior. The guest shares experiences of surprising findings, emphasizing the need for humility and continuous testing. Understanding counterfactual outcomes remains crucial in navigating the complexities of causal inference and effectively communicating results to stakeholders.
Embracing Counterfactual Thinking
The guest delves into the cognitive demands of counterfactual thinking, stressing the necessity of considering alternate scenarios and outcomes. While acknowledging the complexity of causal inference, the guest discusses the challenges faced in conceptualizing different decision paths simultaneously. Counterfactual thinking plays a key role in evaluating models, understanding data discrepancies, and adapting strategies in response to dynamic environments.
Impactful Strategies for Recommendations
Exploring how models cater to human psychology, the podcast uncovers the iterative nature of experimentation and the need for robust solutions in recommendation systems. The emphasis is on the dynamic and inconsistent nature of human behavior, challenging traditional model evaluation methods. By addressing the evolving landscape of business needs and customer expectations, the team incorporates causal inference to optimize user experiences and drive growth.
Understanding Causal Recommender Systems and Uplift Modeling
Exploring the development of causal recommender systems and uplift modeling involves considering various treatments and their effects. By assessing different metrics and optimizing outcomes based on potential impact, such systems aim to determine the most suitable segments for specific treatments. Dealing with biased data requires techniques like propensity weighting to debias data for accurate analysis. Utilizing techniques such as prospective estimation can help in understanding individual outcomes and optimizing treatment allocations.
Importance of Keeping Solutions Simple and Impactful
The importance of simplicity and impact in problem-solving is highlighted, drawing parallels with early experiences in designing a game. Emphasizing the need to start with simple solutions that may not be optimal but are effective, the journey to finding the right balance involves incremental complexity. By focusing on optimizing promotions and understanding user behavior patterns, the process evolves into a multi-lever system. Learning from mistakes and continuously evaluating outcomes is crucial, guiding the development of more sophisticated strategies for impactful decision-making.
Video version of this episode is available here Causal personalization? Dima did not love computers enough to forget about his passion for understanding people.
His work at Booking.com focuses on recommender systems and personalization, and their intersection with AB testing, constrained optimization and causal inference.
Dima's passion for building things started early in his childhood and continues up to this day, but recent events in his life also bring new opportunities to learn.
In the episode, we discuss:
What can we learn about human psychology from building causal recommender systems?
What it's like to work in a culture of radical experimentation?
Why you should not skip your operations research classes?
Ready to dive in?
About The Guest Dima Goldenberg is a Senior Machine Learning Manager at Booking.com, Tel Aviv, where he leads machine learning efforts in recommendations and personalization utilizing uplift modeling. Dima obtained his MSc in Tel Aviv University and currently pursuing PhD on causal personalization at Ben Gurion University of the Negev. He led multiple conference workshops and tutorials on causality and personalization and his research was published in top journals and conferences including WWW, CIKM, WSDM, SIGIR, KDD and RecSys.
About The Host Aleksander (Alex) Molak is an independent machine learning researcher, educator, entrepreneur and a best-selling author in the area of causality (https://amzn.to/3QhsRz4).
