Recsperts - Recommender Systems Experts

In episode number eight of Recsperts we discuss music recommender systems, the meaning of artist fairness and perspectives on recommender evaluation. I talk to Christine Bauer, who is an assistant professor at the University of Utrecht and co-organizer of the PERSPECTIVES workshop. Her research deals with context-aware recommender systems as well as the role of fairness in the music domain. Christine published work at many conferences like CHI, CHIIR, ICIS, and WWW.In this episode we talk about the specifics of recommenders in the music streaming domain. In particular, we discuss the interests of different stakeholders, like users, the platform, or artists. Christine Bauer presents insights from her research on fairness with respect to the representation of artists and their interests. We talk about gender imbalance and how recommender systems could serve as a tool to counteract existing imbalances instead of reinforcing them, for example with simulations and reranking. In addition, we talk about the lack of multi-method evaluation and how open datasets incline researchers to focus too much on offline evaluation. In contrast, Christine argues for more user studies and online evaluation.We wrap up with some final remarks on context-aware recommender systems and the potential of sensor data for improving context-aware personalization.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode:Website of Christine BauerChristine Bauer on LinkedInChristine Bauer on TwitterPERSPECTIVES 2022: Perspectives on the Evaluation of Recommender Systems5th FAccTRec Workshop: Responsible RecommendationPapers:Ferraro et al. (2021): What is fair? Exploring the artists' perspective on the fairness of music streaming platformsFerraro et. al (2021): Break the Loop: Gender Imbalance in Music RecommendersJannach et al. (2020): Escaping the McNamara Fallacy: Towards More Impactful Recommender Systems ResearchBauer et al. (2015): Designing a Music-controlled Running Application: a Sports Science and Psychological PerspectiveDey et al. (2000): Towards a Better Understanding of Context and Context-AwarenessGeneral Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/ (03:18) - Introducing Christine Bauer (09:08) - Multi-Stakeholder Interests in Music Recommender Systems (15:56) - Context-Aware Music Recommendations (21:55) - Fairness in Music RecSys (41:22) - Trade-Offs between Fairness and Relevance (48:18) - Evaluation Perspectives (01:02:37) - Further RecSys Challenges

In episode number seven, we meet Jacopo Tagliabue and discuss behavioral testing for recommender systems and experiences from ecommerce. Before Jacopo became the director of artificial intelligence at Coveo, he had founded tooso, which was later acquired by Coveo. Jacopo holds a PhD in cognitive intelligence and made many contributions to conferences like SIGIR, WWW, or RecSys. In addition, he serves as adjunct professor at NYU.In this episode we introduce behavioral testing for recommender systems and the corresponding framework RecList that was created by Jacopo and his co-authors. Behavioral testing goes beyond pure retrieval accuracy metrics and tries to uncover unintended behavior of recommender models. RecList is an adaption of CheckList that applies behavioral testing to NLP and which was proposed by Microsoft some time ago. RecList comes with an open-source framework with ready set datasets for different recommender use-cases like similar, sequence-based and complementary item recommendations. Furthermore, it offers some sample tests to make it easier for newcomers to get started with behavioral testing. We also briefly touch on the upcoming CIKM data challenge that is going to focus on the evaluation of recommender systems.In the end of this episode Jacopo also shares his insights from years of building and using diverse ML Ops tools and talk about what he refers to as the "post-modern stack".Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode:Jacopo Tagliabue on LinkedInGitHub: RecListCIKM RecEval Analyticup 2022 (sign up!)GitHub: You Don't Need a Bigger Boat - end-to-end (Metaflow-based) implementation of an intent prediction (and session recommendation) flowCoveo SIGIR eCOM 2021 Data Challenge DatasetBlogposts: The Post-Modern Stack - Joining the modern data stack with the modern ML stackTensorFlow RecommendersTorchRecNVIDIA MerlinRecommenders (by Microsoft)recbolePapers:Chia et al. (2022): Beyond NDCG: behavioral testing of recommender systems with RecListRibeiro et al. (2020): Beyond Accuracy: Behavioral Testing of NLP models with CheckListBianchi et al. (2020): Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop ScenarioGeneral Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/

In episode number six, we welcome Manel Slokom to the show and talk about purpose-aware privacy-preserving data for recommender systems. Manel is a 4th year PhD student at Delft University of Technology. For three years in a row she served as student volunteer at RecSys - before becoming student volunteer co-chair herself in 2021. Besides working on privacy and fairness, she also dedicates herself to simulation and in particular synthetic data for recommender systems - also co-organizing the 1st SimuRec Workshop as part of RecSys 2021.This episode is definitely worth a longer run. Manel and I discussed fairness and privacy in recommender systems and how ratings can leak signals about sensitive personal information. For example, classifiers may exploit ratings in order to effectively determine one's gender. She explains "Personalized Blurring", which is the approach she developed to personalize gender obfuscation in user rating data, as well as how this can contribute to more diverse recommendations.In our discussion, we also touch "data-centric AI", a term recently formulated by Andrew Ng, and how adapting feedback data may yield underestimated effects on recommendations that can lead to "data-centric recommender systems". In addition, we dived into the differences between simulated and synthetic data which brought us to the SimuRec workshop that she co-organized as part of RecSys 2021.Finally, Manel provides some recommendations for young researcher to become active RecSys community members and benefit from exchange: talk to people and volunteer at RecSys.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode:Manel on TwitterManel on LinkedInManel at TU Delft (find more papers referenced there)SimuRec Workshop at RecSys 2021FAccTrec Workshop at RecSys 2021Andrew Ng: Unbiggen AI (from IEEE Spectrum)Papers:Slokom et al. (2021): Towards user-oriented privacy for recommender system data: A personalization-based approach to gender obfuscation for user profilesWeinsberg et al. (2012): BlurMe: Inferring and Obfuscating User Gender Based on RatingsEkstrand et al. (2018): All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and EffectivenessSlokom et al. (2018): Comparing recommender systems using synthetic dataBurke et al. (2018): Synthetic Attribute Data for Evaluating Consumer-side FairnessBurke et al. (2005): Identifying Attack Models for Secure RecommendationNarayanan et al. (2008): Robust De-anonymization of Large Sparse DatasetsGeneral Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/

In episode five my guest is Zeno Gantner, who is a principal applied scientist at Zalando. Zeno obtained his PhD from the University of Hildesheim where he was investigating ML-based recommender systems. As a principal applied scientist he is responsible for strategy, mentoring and setting standards for different initiatives on fashion recommendations impacting over 48 million customers in Europe.We discuss the ramifications and limitations of positive-only implicit feedback, touch on how reinforcement learning and more rating-like feedback can help as well as how to treat multiple feedback levels. In the main part, we turn our focus towards fashion recommendations and the “usual suspects” of typical e-commerce recommender systems.  We also discuss the goal of creating more fashion-specific recommendations and making users come back for inspiration. This involves a lot of domain-specific modeling and design of experiences to cater the needs for various user segments: from fashionistas to pragmatic customers. This also involves putting users into the “driver seat” of recommenders as well as understanding how to achieve long-term customer satisfaction.Finally, we briefly touch on the topic of size and fit recommendations and finish with an outlook on the future developments leading to fashion recommendations becoming its own subfield within the recommender systems space.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode:Preferably reach out to Zeno Gantner via email (find his address mentioned by the end of the episode)Fashion DNA by Zalando Research (Paper)Fashion MNIST (image dataset)Workshop on Recommender Systems in Fashion 2021RecSys Challenge 2022 on Session-based Fashion Item Recommendation by DressipiH&M Personalized Fashion Recommendation Challenge on KaggleSpotify: A Product Story - Episode 4: Human vs MachineDataset for trivago RecSys Challenge 2019RecSys 2020: Tutorial on Conversational Recommender SystemsPapers:Rendle et al. (2009): Bayesian Personalized Ranking from Implicit Feedback (2009)Loni et al. (2016): Bayesian Personalized Ranking with Multi-Channel User FeedbackSheikh et al. (2019): A Deep Learning System for Predicting Size and Fit in Fashion E-CommerceWilhelm et al. (2018): Practical Diversified Recommendations on YouTube with Determinantal Point ProcessesGeneral Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/

Felice Merra, an applied scientist at Amazon, discusses Adversarial Machine Learning in Recommender Systems. Topics include perturbing data and model parameters, defense strategies, motivations for attacks, and privacy-preserving learning. The goal is to make systems more robust against potential attacks. They also touch on the challenges of robustifying multimedia recommender systems.

In episode three I am joined by Olivier Jeunen, who is a postdoctoral scientist at Amazon. Olivier obtained his PhD from University of Antwerp with his work "Offline Approaches to Recommendation with Online Success". His work concentrates on Bandits, Reinforcement Learning and Causal Inference for Recommender Systems.We talk about methods for evaluating online performance of recommender systems in an offline fashion and based on rich logging data. These methods stem from fields like bandit theory and reinforcement learning. They heavily rely on simulators whose benefits, requirements and limitations we discuss in greater detail. We further discuss the differences between organic and bandit feedback as well as what sets recommenders apart from advertising. We also talk about the right target for optimization and receive some advice to continue livelong learning as a researcher, be it in academia or industry.Olivier has published multiple papers at RecSys, NeurIPS, WSDM, UMAP, and WWW. He also won the RecoGym challenge with his team from University of Antwerp. With research internships at Criteo, Facebook and Spotify Research he brings significant experience to the table. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode:Olivier's WebsiteOlivier Jeunen on LinkedIn and TwitterSimulators:RecoGymRecSimRecSimNGOpen Bandit PipelineBlogpost: Lessons Learned from Winning the RecoGym ChallengeRecSys 2020 REVEAL Workshop on Bandit and Reinforcement Learning from User InteractionsRecSys 2021 Tutorial on Counterfactual Learning and Evaluation for Recommender SystemsNeurIPS 2021 Workshop on Causal Inference and Machine LearningThesis and Papers:Dissertation: Offline Approaches to Recommendation with Online SuccessChen et al. (2018): Top-K Off-Policy Correction for a REINFORCE Recommender SystemJeunen et al. (2021): Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved ConfoundersJeunen et al. (2021): Top-𝐾 Contextual Bandits with Equity of ExposureGeneral Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/

In episode two I am joined by Even Oldridge, Senior Manager at NVIDIA, who is leading the Merlin Team. These people are working on an open-source framework for building large-scale deep learning recommender systems and have already won numerous RecSys competitions.We talk about the relevance and impact of deep learning applied to recommender systems as well as the challenges and pitfalls of deep learning based recommender systems. We briefly touch on Even's early data science contributions at PlentyOfFish, a Canadian online-dating platform. Starting with personalized recommendations of people to people he transitioned to realtor, a real-estate marketplace. From the potentially biggest social decision in life to the probably biggest financial decision in life he has really been involved with recommender systems at the extremes. At NVIDIA - to which he refers as the one company that works with all the other AI companies - he pushes for Merlin as large-scale, accessible and efficient platform for developing and deploying recommender systems on GPUs.This brought him also closer to the community which he served as industry Co-Chair at RecSys in 2021 as well as to winning multiple RecSys competitions with his team in the recent years.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode:Even Oldridge on LinkedIn and TwitterNVIDIA MerlinNVIDIA Merlin at GitHubEven's upcoming Talk at GTC 2021: Building and Deploying Recommender Systems Quickly and Easily with NVIDIA MerlinPlentyOfFish, realtorfast.aiTwitter RecSys Challenge 2021Recommending music on Spotify with Deep LearningPapersDacrema et al. (2019): Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches (best paper award at RecSys 2019)Jannach et al. (2020): Why Are Deep Learning Models Not Consistently Winning Recommender Systems Competitions Yet?: A Position PaperMoreira et al. (2021): Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based RecommendationDeotte et al. (2021): GPU Accelerated Boosted Trees and Deep Neural Networks for Better Recommender Systems General Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/

In this first interview we talk to Kim Falk, Senior Data Scientist, multiple RecSys Industry Chair and author of the book "Practical Recommender Systems". We introduce into recommenders from a practical perspective discussing the fundamental difference between content-based and collaborative filtering as well as the cold-start problem - no mathematical deep-dive yet, but expect it to follow. In addition, we reason what constitutes good recommendations and briefly touch on a couple of ways of finding that out.Looking a bit into the history of the recommender systems community, we touch on the Netflix Prize that was running from 2006 to 2009 as well as on the RecSys - the leading conference in recommender systems, where we also met for the first time.In the end, we discuss a couple of challenges the field faces, in particular associated with approaches based on deep learning. Besides that, Spiderman will accompany our conversation at certain times. Plus many practical recommendations included on how to get started. Stay tuned!Links from this Episode:Kim Falk on LinkedIn and TwitterBook: Practical Recommender Systems (Manning) (get 37% discount with the code podrecsperts37 during checkout)GitHub Repository for PRS BookACM Conference on Recommender Systems 2021 (Amsterdam)Recommender Systems Specialization at CourseraAmazon.com Recommendations: Item-to-Item Collaborative FilteringNetflix PrizeNetflix Prize dataset on KaggleNew York Times: A $1 Million Research Bargain for Netflix, and Maybe a Model for OthersEvaluation Measures for Information RetrievalPaper by Dacrema et al. (2019): Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches (best paper award at RecSys 2019)Recommending music on Spotify with Deep LearningMovieLens RecommendersGeneral Links:Follow me on Twitter: https://twitter.com/LivesInAnalogiaSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/Twitter and LinkedIn posts for sharing:LinkedInTwitter

Have you ever though about how Spotify is able to generate its fantastic Discover Weekly Playlist, how Amazon is generating a fortune by showing what other like you purchased in the past, or how Netflix achieves high user retention? The answer is personalization and in this show we focus on the most prominent way to achieve personalization: recommender systems.Whether you are a beginner and new to the field or you have already build recommenders, this show is to bring you the experts in recommender systems to share their knowledge and expertise with all of us. It is for making the topic more accessible and to provide a regular coverage of basics and advances in recommender systems research and application. I invite the experts to share their insights and to provide you with the right knowledge to get started and gain expertise yourself.In this introductory episode I am going to share some exemplary use cases from different industries (music streaming, e-commerce, travel, or social networks) along with challenges and problems in research and application. Plus, I am presenting the first guest for our upcoming episode.Links from the show:ACM Conference on Recommender Systems 2021 (Amsterdam): https://recsys.acm.org/recsys21/Introductory Python RecSys Training: https://github.com/mkurovski/recsys_trainingFollow me on Twitter: https://twitter.com/LivesInAnalogiaRead my RecSys Blogposts: https://medium.com/@marcel.kurovskiSend me your comments, questions and suggestions to marcel@recsperts.comPodcast Website: https://www.recsperts.com/