Gottam Sai Bhrath, a Senior Machine Learning Engineer, and Cole Bailey, an ML Platform Engineering Manager at Delivery Hero, dive into the intricacies of optimizing machine learning practices across their global operations. They discuss the evolution of feature stores, balancing centralized and decentralized models, and overcoming technological integration challenges in a diverse organization. The conversation highlights their collaborative approach to building a Global Feature Store, addressing real-time data processing and strategies to maintain data integrity in a complex environment.
Read more
AI Summary
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Delivery Hero has shifted to a decentralized MLOps approach, empowering local teams to innovate solutions tailored to their specific contexts.
The initiative to create a global feature store emphasizes collaboration across departments to improve data sharing and operational efficiency.
Delivery Hero's inner sourcing strategy promotes resource optimization and collaboration, addressing resource constraints while maximizing engineering output across diverse teams.
Deep dives
Transition from Centralized to Decentralized Teams
Delivery Hero has transitioned from a centralized machine learning platform team to a more decentralized approach where individual local teams are encouraged to develop solutions tailored to their specific needs. Initially, the centralized model faced challenges due to varied demands from multiple teams, leading to difficulties in prioritizing projects effectively. As a response, the organization began empowering local teams to address their unique problems while collaborating on solutions that could have global applications. This approach not only enhances focus on localized issues but also fosters innovation and builds a more cooperative environment across diverse departments.
Building a Global Feature Store
The development of a global feature store is a primary focus for Delivery Hero, aiming to integrate various feature store implementations from different departments to streamline processes. Previous experiences with isolated feature stores highlighted the need for a collaborative effort to create an easily accessible and scalable solution. By conducting workshops with representatives from multiple teams, the company has aimed to solidify a shared vision while addressing the distinct needs of various departments, ensuring that all teams can effectively utilize the new feature store. This collaborative architecture is anticipated to alleviate operational silos and stimulate enhanced data sharing and interoperability.
Importance of Clear Definitions and Communication
During the collaborative workshops for the feature store initiative, a crucial challenge was achieving consensus on fundamental terminologies and definitions surrounding features and feature stores. Differing interpretations among teams often led to misunderstandings and complications, emphasizing the need for clear, agreed-upon frameworks. Establishing a unified language helps streamline discussions and align objectives, which is essential for successfully implementing cross-departmental solutions. By prioritizing communication and clarity, Delivery Hero aims to mitigate the confusion that typically accompanies collaborative technical projects.
Adapting to Resource Constraints in a Changing Economy
The recent shift in the tech landscape has necessitated a strategic approach to handling resource constraints, particularly in hiring and expanding engineering teams. Delivery Hero has moved away from previous practices of rapidly hiring engineers to solve problems, instead fostering a culture of inner sourcing where departments can share and leverage existing resources and expertise. This strategy allows teams to streamline operations and maximize output despite budgetary constraints, promoting collaboration and innovation within a diverse organization. Such adaptability ensures that teams remain effective, even as market dynamics evolve.
Future Vision for Feature Store Integration
The future vision for Delivery Hero's global feature store is to create a comprehensive and adaptable environment that meets the diverse needs of various departments while ensuring ease of access and integration. As teams focus on aligning their existing solutions with the global framework, the goal is to enhance features incrementally based on departmental requirements, proving the concept's value through tangible results. This ongoing development will allow for scalability and flexibility in adopting new technologies, ultimately driving efficiencies across the organization. As the project matures, it is expected that this unified feature store will become an indispensable asset for the company.
Global Feature Store: Optimizing Locally and Scaling Globally at Delivery Hero // MLOps Podcast #263 with Delivery Hero's Gottam Sai Bharath, Senior Machine Learning Engineer & Cole Bailey, ML Platform Engineering Manager.
// Abstract
Delivery Hero innovates locally within each department to develop MLOps practices most effective in that particular context. We also discuss our efforts to reduce redundancy and inefficiency across the company. Hear about our experiences in creating multiple micro feature stores within our departments, and our goal to unify these into a Global Feature Store that is more powerful when combined.
// Bio
Sai Bharath Gottam
With a passion for translating complex technical concepts into practical solutions, Sai excels at making intricate topics accessible and engaging. As a Senior Machine Learning Engineer at Delivery Hero, Sai works on cutting-edge machine learning platforms that guarantee seamless delivery experiences. Always eager to share insights and innovations, Sai is committed to making technology understandable and enjoyable for all.
Cole Bailey
Bridging data science and production-grade software engineering.
// MLOps Jobs board
https://mlops.pallet.xyz/jobs
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Website: https://www.deliveryhero.com/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Sai on LinkedIn: https://www.linkedin.com/in/sai-bharath-gottam/
Connect with Cole on LinkedIn: www.linkedin.com/in/cole-bailey
Timestamps:
[00:00] Sai and Cole's preferred coffee
[00:42] Takeaways
[01:51] Please like, share, leave a review, and subscribe to our MLOps channels!
[02:08] Life changes in Delivery Hero
[05:21] Global Feature Store and Pandora
[12:21] Tech integration strategies
[20:08] Defining Feature and Feature Store
[22:46] Feature Store vs Data Platform
[26:26] Features are discoverable
[32:56] Onboarding and Feature Testing
[36:00] Data consistency
[41:07] Future Vision Feature Store
[44:17] Multi-cloud strategies
[46:33] Wrap up
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode