Integrating ML Evaluation into CI/CD Pipelines and Challenges with Client Approval

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

Syed Asad is an Innovator, Generative AI & Machine Learning Engineer, and a Champion for Ethical AI
MLOps podcast #233 with Syed Asad, Lead AI/ML Engineer at KiwiTech // Retrieval Augmented Generation. A big thank you to @ for sponsoring this episode! AWS - // Abstract Everything and anything around RAG. // Bio Currently Exploring New Horizons: Syed is diving deep into the exciting world of Semantic Vector Searches and Vector Databases. These innovative technologies are reshaping how we interact with and interpret vast data landscapes, opening new avenues for discovery and innovation. Specializing in Retrieval Augmented Generation (RAG): Syed's current focus also includes mastering Retrieval Augmented Generation Techniques (RAGs). This cutting-edge approach combines the power of information retrieval with generative models, setting new benchmarks in AI's capability and application. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://sanketgupta.substack.com/ Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584 Sanket's blogs on Medium in the past: https://medium.com/@sanket107 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Syed on LinkedIn: https://www.linkedin.com/in/syed-asad-76815246/ Timestamps: [00:00] Syed's preferred coffee [00:31] Takeaways [03:17] Please like, share, leave a review, and subscribe to our MLOps channels! [03:37] A production issue [07:37] CSV file handling risks [09:42] Embedding models not suitable [11:22] Inference layer experiments and use cases [14:00] AWS service handling the issue [17:35] Salad testing and insights [22:12] OpenAI vs Customization [24:30] Difference between Olama and VLLM [27:16] Fine-tuning of small LLMs [29:51] Evaluation framework [32:04] MLOps for efficient ML [37:12] Determining the pricing of tools [39:35] Manage Dependency Risk [40:27] Get in touch with Syed on LinkedIn [41:46] ML Engineers are now all AI Engineers [43:01] The hard framework [43:53] Wrap up