Limitations and Excitement for Transformers in NLP

MLOps podcast #180 with Sachin Abeywardana Deep Learning Engineer at Canva AI, Adventures in Building CLIP and Other (Largeish) Language Models sponsored by Prem AI. // Abstract Sachin takes us on an adventure, sharing insights on the pitfalls of not understanding the broader product and the importance of incorporating AI and machine learning capabilities. From the use of AI models to grammar correction and code generation to the fascinating Clip model and the challenges of balancing work and family life, this episode promises to be both informative and thought-provoking. // Bio Sachin is the father of two beautiful children. He completed his PhD in Bayesian Machine Learning at University of Sydney in 2015. In 2016 he discovered Deep Learning and hasn't looked back. He currently works as a Senior Machine Learning Engineer at Canva and is mainly focusing on NLP problems. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Sachin Blogs: https://sachinruk.github.io/blog.html https://sachinruk.github.io/blog/ Graph ML link: http://web.stanford.edu/class/cs224w/ ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Sachin on LinkedIn: https://www.linkedin.com/in/sachinabeywardana/ Timestamps: [00:00] Sachin's preferred beverage [00:26] Takeaways [02:30] Chat GPT user [05:58] Understanding on reliable Agents [08:10] Sachin's background [12:45] Staying at Deep Learning [16:17] Recommendation or Lead Scoring [17:36] Vector database [19:00] Sachin's blogs [23:26] The cap people [26:10] Pursuing business case [27:33] Canva [31:16] Incorporating AI and Machine Learning [32:17] Sponsor Ad [38:22] Eliminating unnecessary steps [39:00] Interacting with the product team [43:04] Criticisms on the current architecture limitations [45:58] Insufficient exploration of Transformers [47:42] Explaining GraphML [52:35] Fine-tuning ChatGPT2 [57:54] Leading ML Engineers and teams [59:40] Being practical with Math [1:05:52] Wrap up