Syed Asad, an Innovator and AI Engineer, discusses Retrieval Augmented Generation (RAG), Semantic Vector Searches, and Vector Databases reshaping data landscapes. Topics include AI model deployment complexities, AI evaluation frameworks, challenges in client approval, and struggles with data ingestion in AI environments.
Read more
AI Summary
Highlights
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Retrieval Augmented Generation combines information retrieval with generative models for AI advancement.
Testing methodologies are crucial for AI tool deployment in production, emphasizing iteration and customization.
Deep dives
In-depth Discussion on AI Tools and Production Challenges
The podcast delves into a detailed discussion with Asad, a senior AI engineer at Kiwi Tech, exploring the complexities and challenges faced in AI tool deployment in a production environment. Asad candidly shares his experiences with various open source projects and tools, emphasizing the importance of thorough testing and iteration. He highlights the significance of proper testing methodologies, especially in scenarios where quick deployment is crucial.
Struggles with Handling Large Data Sets and Vector Embeddings
Asad narrates a production issue involving a sizable CSV file and the challenges faced in converting it into vector embeddings for efficient retrieval. Despite attempts using different models and tools like Pandas AI, constraints in processing the data arise due to the file size and complexity. Ultimately, a solution involving converting the CSV file into a Pocket format greatly reduces the size and enhances the query response speed.
Exploration of Inference Layer and Model Selection
The conversation with Asad extends to exploring the inference layer in AI models and the importance of careful selection and testing of frameworks and tools. Discussions revolve around leveraging research teams to assess different frameworks, such as FastEmbed, for specific scenarios. Asad emphasizes the need for a customized approach towards tool selection, acknowledging the pros and cons of utilizing established models versus developing personalized frameworks.
Evaluation of AI Services and Tools for Production Environments
Asad shares insights on evaluating AI services like Salad and Base10, emphasizing factors like affordability, ease of deployment, and operational challenges. The conversation also touches upon the significance of assessing tools like Orpo and third-party services such as Onslat AI for fine-tuning LLMs, streamlining evaluation frameworks, and optimizing production workflows. Asad highlights the importance of a structured approach towards evaluating and integrating AI tools into production environments.
Syed Asad is an Innovator, Generative AI & Machine Learning Engineer, and a Champion for Ethical AI
MLOps podcast #233 with Syed Asad, Lead AI/ML Engineer at KiwiTech // Retrieval Augmented Generation.
A big thank you to @ for sponsoring this episode! AWS -
// Abstract
Everything and anything around RAG.
// Bio
Currently Exploring New Horizons:
Syed is diving deep into the exciting world of Semantic Vector Searches and Vector Databases. These innovative technologies are reshaping how we interact with and interpret vast data landscapes, opening new avenues for discovery and innovation.
Specializing in Retrieval Augmented Generation (RAG):
Syed's current focus also includes mastering Retrieval Augmented Generation Techniques (RAGs). This cutting-edge approach combines the power of information retrieval with generative models, setting new benchmarks in AI's capability and application.
// MLOps Jobs board
https://mlops.pallet.xyz/jobs
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Website: https://sanketgupta.substack.com/
Our paper on this topic "Generalized User Representations for Transfer Learning": https://arxiv.org/abs/2403.00584
Sanket's blogs on Medium in the past: https://medium.com/@sanket107
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Syed on LinkedIn: https://www.linkedin.com/in/syed-asad-76815246/
Timestamps:
[00:00] Syed's preferred coffee
[00:31] Takeaways
[03:17] Please like, share, leave a review, and subscribe to our MLOps channels!
[03:37] A production issue
[07:37] CSV file handling risks
[09:42] Embedding models not suitable
[11:22] Inference layer experiments and use cases
[14:00] AWS service handling the issue
[17:35] Salad testing and insights
[22:12] OpenAI vs Customization
[24:30] Difference between Olama and VLLM
[27:16] Fine-tuning of small LLMs
[29:51] Evaluation framework
[32:04] MLOps for efficient ML
[37:12] Determining the pricing of tools
[39:35] Manage Dependency Risk
[40:27] Get in touch with Syed on LinkedIn
[41:46] ML Engineers are now all AI Engineers
[43:01] The hard framework
[43:53] Wrap up
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode