Recsperts - Recommender Systems Experts #29: Transformers for Recommender Systems with Craig Macdonald and Sasha Petrov
In episode 29 of Recsperts, I welcome Craig Macdonald, Professor of Information Retrieval at the University of Glasgow, and Aleksandr “Sasha” Petrov, PhD researcher and former applied scientist at Amazon. Together, we dive deep into sequential recommender systems and the growing role of transformer models such as SASRec and BERT4Rec.
Our conversation begins with their influential replicability study of BERT4Rec, which revealed inconsistencies in reported results and highlighted the importance of training objectives over architecture tweaks. From there, Craig and Sasha guide us through their award-winning research on making transformers for sequential recommendation with large corpora both more effective and more efficient. We discuss how recency sampling (RSS) reduces training times dramatically, and how gSASRec overcomes the problem of overconfidence in models trained with negative sampling. By generalizing the sigmoid function (gBCE), they were able to reconcile cross-entropy–based optimization results with negative sampling, matching the effectiveness of softmax approaches while keeping training scalable for large corpora.
We also explore RecJPQ, their recent work on joint product quantization for item embeddings. This approach makes transformer-based sequential recommenders substantially faster at inference and far more memory-efficient for embeddings—while sometimes even improving effectiveness thanks to regularization effects. Towards the end, Craig and Sasha share their perspective on generative approaches like GPTRec, the promises and limits of large language models in recommendation, and what challenges remain for the future of sequential recommender systems.
Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts.
Don’t forget to follow the podcast and please leave a review.
- (00:00) - Introduction
- (04:09) - About Craig Macdonald
- (04:46) - About Sasha Petrov
- (13:48) - Tutorial on Transformers for Sequential Recommendations
- (19:24) - SASRec vs. BERT4Rec
- (21:25) - Replicability Study of BERT4Rec for Sequential Recommendation
- (32:52) - Training Sequential RecSys using Recency Sampling
- (40:01) - gSASRec for Reducing Overconfidence by Negative Sampling
- (01:00:51) - RecJPQ: Training Large-Catalogue Sequential Recommenders
- (01:21:37) - Generative Sequential Recommendation with GPTRec
- (01:29:12) - Further Challenges and Closing Remarks
Links from the Episode:
- Craig Macdonald on LinkedIn
- Sasha Petrov on LinkedIn
- Sasha's Website
- Tutorial: Transformers for Sequential Recommendation (ECIR 2024)
- Tutorial Recording from ACM European Summer School in Bari (2024)
- Talk: Neural Recommender Systems (European Summer School in Information Retrieval 2024)
Papers:
- Kang et al. (2018): Self-Attentive Sequential Recommendation
- Sun et al. (2019): BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
- Petrov et al. (2022): A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation
- Petrov et al. (2022): Effective and Efficient Training for Sequential Recommendation using Recency Sampling
- Petrov et al. (2024): RSS: Effective and Efficient Training for Sequential Recommendation Using Recency Sampling (extended version)
- Petrov et al. (2023): gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling
- Petrov et al. (2025): Improving Effectiveness by Reducing Overconfidence in Large Catalogue Sequential Recommendation with gBCE loss
- Petrov et al. (2024): RecJPQ: Training Large-Catalogue Sequential Recommenders
- Petrov et al. (2024): Efficient Inference of Sub-Item Id-based Sequential Recommendation Models with Millions of Items
- Rajput et al. (2023): Recommender Systems with Generative Retrieval
- Petrov et al. (2023): Generative Sequential Recommendation with GPTRec
- Petrov et al. (2024): Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learning
General Links:
- Follow me on LinkedIn
- Follow me on X
- Send me your comments, questions and suggestions to marcel.kurovski@gmail.com
- Recsperts Website
Disclaimer:
Craig holds concurrent appointments as a Professor of Information Retrieval at University of Glasgow and as an Amazon Scholar. This podcast describes work performed at the University of Glasgow and is not associated with Amazon.