Evaluating and Integrating ML Models // Morgan McGuire and Anish Shah // #213
MLOps.community
User Data Gathering and Evaluation Frameworks
Discussion on integrating user data into evaluation frameworks for LM-based applications, clustering tactics for support topics, updating documentation based on user queries, and the challenges of evaluating machine learning models with non-transparent data sources.
Morgan McGuire has held a variety of roles in the past 13 years. In 2008, he completed a Research Internship at Queen Mary, University of London. Currently, he is the Head of Growth ML and Growth ML Engineer at Weights & Biases. Anish Shah has been working in the tech industry since 2015. In 2015, he was a Technical Support at Fox School of Business at Temple University. In 2021, he has been an MLOps Engineer - Growth and a Tier 2 Support Machine Learning Engineer at Weights & Biases. ______________________________________________ Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them? In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output, and debugging. You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches. Join the AI in Production Conference on February 22 here: https://home.mlops.community/home/events/ai-in-production-2024-02-15 ______________________________________________ MLOps podcast #213 with Weights and Biases' Growth Director, Morgan McGuire and MLE, Anish Shah, Evaluating and Integrating ML Models brought to you by our Premium Brand Partner @WeightsBiases. // Abstract Anish Shah and Morgan McGuire share insights on their journey into ML, the exciting work they're doing at Weights and Biases, and their thoughts on MLOps. They discuss using large language models (LLMs) for translation, pre-written code, and internal support. They discuss the challenges of integrating LLMs into products, the need for real use cases, and maintaining credibility. They also touch on evaluating ML models collaboratively and the importance of continual improvement. They emphasize understanding retrieval and balancing novelty with precision. This episode provides a deep dive into Weights and Biases' work with LLMs and the future of ML evaluation in MLOps. It's a must-listen for anyone interested in LLMs and ML evaluation. // Bio Anish Shah Anish loves turning ML ideas into ML products. He started his career working with multiple Data Science teams within SAP, working with traditional ML, deep learning, and recommendation systems before landing at Weights & Biases. With the art of programming and a little magic, Anish crafts ML projects to help better serve our customers, turning “oh nos” to “a-ha”s! Morgan McGuire Morgan is a Growth Director and an ML Engineer at Weights & Biases. He has a background in NLP and previously worked at Facebook on the Safety team where he helped classify and flag potentially high-severity content for removal. // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI in Production Conference: https://home.mlops.community/home/events/ai-in-production-2024-02-15 Website: https://wandb.ai/ Prompt Templates the Song: https://www.youtube.com/watch?v=g6WT85gIsE8 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Morgan on LinkedIn: https://www.linkedin.com/in/morganmcg1/ Connect with Anish on LinkedIn: https://www.linkedin.com/in/anish-shah/
Remember Everything You Learn from Podcasts
