Ketan Umare, CEO at Union.ai, discusses ML orchestration vs MLOps, prioritizing features for open-source and enterprise projects, challenges of ML orchestration, building a flexible architecture, Unionverse flight and plugin models, challenges of large language models and evaluation.
Read more
AI Summary
Highlights
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Prioritizing feature development between open-source projects and enterprise offerings is crucial for startups.
Designing ML infrastructure to handle the specific requirements of large language models is a challenge in ML Ops.
Deep dives
Building a Startup and Differentiating Features
In this episode, Kaitan discusses his experience building a startup and differentiating features for flight and union. He talks about the challenges of prioritizing feature development between the open-source project and the enterprise offerings, highlighting the importance of catering to the needs of enterprises while still considering the requests from the flight community.
The Community and User Experience
Kaitan emphasizes the significance of the flight community and the user experience. He discusses the community-driven development approach, where flight aims to address the needs and requests of its users. He highlights the community's diversity and the importance of providing support and empathy to users, even if they are new to certain concepts or technologies.
Navigating ML Infrastructure
The conversation delves into the challenges of navigating ML infrastructure, particularly in the context of NLP models like GPT. Kaitan and Stephen discuss the importance of designing ML infrastructure that can handle the specific requirements of large language models, including multi-cluster support, task-level monitoring, and efficient GPU utilization. They also touch on the complexities of evaluating prompts and the need for benchmarks in the NLP domain.
The Future of ML Ops and Evaluation
Kaitan shares his thoughts on the future of ML Ops, emphasizing the importance of human-in-the-loop applications and the augmentation of human capabilities. He highlights the advancements in infrastructure and techniques like quantization and model freezing. They also touch on the challenges of evaluation and the role of benchmarks in assessing the performance of ML models.
MLOps podcast #183 with Ketan Umare, CEO of Union.AI, MLOps vs ML Orchestration co-hosted by Stephen Batifol.
// Abstract
Let's explore the relationship between Union and Flyte, emphasizing the significance of community-driven development and the challenge of balancing feature requests with security considerations. This conversation highlights the importance of real-time data and secure data handling in orchestrating machine learning models. The Flyte community's empathy and support for newcomers underscore the community's value in democratizing machine learning, making it more accessible and efficient for a broader audience.
// Bio
Ketan Umare is the CEO and co-founder at Union.ai. Previously he had multiple Senior roles at Lyft, Oracle, and Amazon ranging from Cloud, Distributed storage, Mapping (map-making), and machine-learning systems. He is passionate about building software that makes engineers' lives easier and provides simplified access to large-scale systems. Besides software, he is a proud father, and husband, and enjoys traveling and outdoor activities.
// MLOps Jobs board
https://mlops.pallet.xyz/jobs
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Website: https://union.ai/
Flyte: https://flyte.org/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/
Connect with Ketan on LinkedIn: https://www.linkedin.com/in/ketanumare/
Timestamps:
[00:00] Ketan's preferred coffee
[01:05] Takeaways
[03:08] Please like, share, and subscribe to our MLOps channels!
[03:15] Shout out to Ketan and UnionAI for sponsoring this episode!
[04:23] Orchestration recent changes
[07:51] Community with Flyte
[11:26] ML orchestration
[15:40] 50/50 is generous
[20:06] Real-time ML
[21:15] Over engineering without benefits
[23:20] Balancing everything
[27:40] Union verse Flyte
[32:52] High value features of Union AI at the back of Flyte
[40:18] Building LLM infrastructure
[45:30] Traditional ML is the whole prompting
[46:46] LLMs to evaluating prompts
[48:55] Wrap up
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode