MLOps.community  cover image

MLOps.community

Latest episodes

undefined
Apr 21, 2022 • 46min

Traversing the Data Maturity Spectrum: A Startup Perspective // Mark Freeman // Coffee Sessions #94

MLOps Coffee Sessions #94 with Mark Freeman, Traversing the Data Maturity Spectrum: A Startup Perspective. // Abstract A lot of companies talk about having ML and being data-driven, but few are there currently and doing it well. If anything, many companies are on the cusp of implementing ML rather than being ML mature.   As a startup, what decisions are we making today to drive data maturity and set us up for success when we further implement ML in the near future. What business cases are we making for leadership buy-in to invest in data infrastructure as compared to product development while we identify product-market-fit. // Bio Mark is a community health advocate turned data scientist interested in the intersection of social impact, business, and technology. His life’s mission is to improve the well-being of as many people as possible through data—especially among those marginalized. Mark received his M.S. from the Stanford School of Medicine where he was trained in clinical research, experimental design, and statistics with an emphasis on observational studies. In addition, Mark is also certified in Entrepreneurship and Innovation from the Stanford Graduate School of Business. He is currently a senior data scientist at Humu where he builds data tools that drive behavior change to make work better. His core responsibilities center around 1) building data products that reach Humu's end users, 2) providing product analytics for the product team, and 3) building data infrastructure and driving data maturity. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Website: humu.com The Informed Company: How to Build Modern Agile Data Stacks that Drive Winning Insights book: https://www.amazon.com/Informed-Company-Cloud-Based-Explore-Understand/dp/1119748003 Fundamentals of Data Engineering book by Joe Reis and Matt Housley: https://www.oreilly.com/library/view/fundamentals-of-data/9781098108298/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Mark on LinkedIn: https://www.linkedin.com/in/mafreeman2/ Timestamps: [00:00] Introduction to Mark Freeman [01:43] Grab your own Apron merch @ https://mlops.community/! [03:41] LinkedIn stardom! [04:40] Followers or connections? [05:31] Leveraging essential information platform [08:56] Investment in time spent on creating and working on a social platform [12:16] Put yourself out there for people to find you [16:33] Data maturity is a spectrum that takes time to traverse [23:43] Maturity of path [28:43] Fundamentals for data products [33:05] Foundational data capabilities [37:32] Value of metrics [41:48] writing reused code timeframe vs working with stakeholders timeframe [44:11] Wrap up   [45:14] Look for Meetups near you!
undefined
Apr 14, 2022 • 52min

Model Monitoring in Practice: Top Trends // Krishnaram Kenthapadi // MLOps Coffee Sessions #93

MLOps Coffee Sessions #93 with Krishnaram Kenthapadi, Model Monitoring in Practice: Top Trends co-hosted by Mihail Eric // Abstract We first motivate the need for ML model monitoring, as part of a broader AI model governance and responsible AI framework, and provide a roadmap for thinking about model monitoring in practice. We then present findings and insights on model monitoring in practice based on interviews with various ML practitioners spanning domains such as financial services, healthcare, hiring, online retail, computational advertising, and conversational assistants. // Bio Krishnaram Kenthapadi is the Chief Scientist of Fiddler AI, an enterprise startup building a responsible AI and ML monitoring platform. Previously, he was a Principal Scientist at Amazon AWS AI, where he led the fairness, explainability, privacy, and model understanding initiatives in the Amazon AI platform. Prior to joining Amazon, he led similar efforts at the LinkedIn AI team and served as LinkedIn’s representative on Microsoft’s AI and Ethics in Engineering and Research (AETHER) Advisory Board. Previously, he was a Researcher at Microsoft Research Silicon Valley Lab. Krishnaram received his Ph.D. in Computer Science from Stanford University in 2006. He serves regularly on the program committees of KDD, WWW, WSDM, and related conferences, and co-chaired the 2014 ACM Symposium on Computing for Development. His work has been recognized through awards at NAACL, WWW, SODA, CIKM, ICML AutoML workshop, and Microsoft’s AI/ML conference (MLADS). He has published 50+ papers, with 4500+ citations and filed 150+ patents (70 granted). He has presented tutorials on privacy, fairness, explainable AI, and responsible AI at forums such as KDD ’18 ’19, WSDM ’19, WWW ’19 ’20 '21, FAccT ’20 '21, AAAI ’20 '21, and ICML '21. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Website: https://cs.stanford.edu/people/kngk/ https://sites.google.com/view/ResponsibleAITutorial https://sites.google.com/view/explainable-ai-tutorial https://sites.google.com/view/fairness-tutorial https://sites.google.com/view/privacy-tutorial --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/ Connect with Krishnaram on LinkedIn: https://www.linkedin.com/in/krishnaramkenthapadi Timestamps: [00:00] Introduction to Krishnaram Kenthapadi [02:22] Takeaways [04:55] Thank you Fiddler AI for sponsoring this episode! [05:15] Struggles in Explainable AI [06:16] Attacking the problem of difficult models and architectures in Explainability [08:30] Explainable AI prominence [09:56] Importance of password manager and actual security [14:27] Role of Education in Explainable AI systems [18:52] Highly regulated domains in other sectors [21:12] First machine learning wins [23:36] Model monitoring [25:35] Interests in ML monitoring and Explainability [27:27] Future of Explainability in the wide range of ML models [29:57] Non-technical stakeholders' voice [33:54] Advice to ML practitioners to address organizational concerns [38:49] Ethically sourced data set  [42:15] Crowd-sourced labor [43:35] Recommendations to organizations about their minimal explainable product [46:29] Tension in practice [50:09] Wrap up
undefined
Apr 11, 2022 • 42min

Building the World's First Data Engineering Conference // Pete Soderling // MLOps Coffee Sessions #92

MLOps Coffee Sessions #92 with Pete Soderling, Building the World's First Data Engineering Conference. // Abstract Keep things centered around community building and what he looks for in teams. Folks that are building their community around their tool, what advice do you have for that? What's worth turning into a company? // Bio Pete Soderling is the founder of Data Council and the Data Community Fund. As a former software engineer, repeat founder, and investor in more than 40 data-oriented startups, Pete’s lifetime goal is to help 1,000 engineers start successful companies. Most importantly, Pete is a community builder — from his earliest days of working with the data engineering community starting in 2013, he has witnessed the unique power of specialized networks to bring inspiration, knowledge, and support to technical professionals. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Website: datacouncil.ai Youtube channel: https://www.youtube.com/c/DataCouncil --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Pete on LinkedIn: https://www.linkedin.com/in/petesoder/ Timestamps: [00:00] Introduction to Pete Soderling [04:35] Top takeaways from the World's First Data Engineering Conference [05:41] Buzz around the conference [06:37] Intro to Data Council [09:20] Pete's mission statement with investing [11:19] Evaluating gaps in the market and who should solve those [14:45] One company Peter regrets not investing in [16:41] Repeating the same mistake [20:07] Recommendations to engineers to become entrepreneurs [23:30] Questions to consider before investing [27:37] Things to do and avoid in open-source projects [31:03] Something popular you disagree [35:29] Code as an artifact [39:16] Hypothetical fundraising   [40:53] Wrap up
undefined
Apr 7, 2022 • 40min

The Shipyard: Lessons Learned While Building an ML Platform / Automating Adherence // Joseph Haaga // Coffee Sessions #91

MLOps Coffee Sessions #91 with Joseph Haaga, The Shipyard: Lessons Learned While Building an ML Platform / Automating Adherence. // Abstract Joseph Haaga and the Interos team walk us through their design decisions in building an internal data platform. Joseph talks about why their use case wasn't a fit for off the self solutions, what their internal tool snitch does, and how they use git as a model registry.   Shipyard blogpost series: https://medium.com/interos-engineering. // Bio Joseph leads the ML Platform team at Interos, the operational resilience company. He was introduced to ML Ops while working as a Senior Data Engineer and has spent the past year building a platform for experimentation and serving. He lives in Washington, DC, with his dog Cheese. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Website: https://joehaaga.xyz Medium: https://medium.com/interos-engineering Shipyard blogpost series: https://medium.com/interos-engineering --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Joseph on LinkedIn: https://www.linkedin.com/in/joseph-haaga/ Timestamps: [00:00] Introduction to Joseph Haaga [02:07] Please subscribe, follow, like, rate, review our Spotify and Youtube channels [02:31] New! Best of Slack Weekly Newsletter [03:03] Interos [04:33] Global supply chain [05:45] Machine Learning use cases of Interos [06:17] Forecasting and optimization of routes [07:14] Build, buy, open-source decision making [10:06] Experiences with Kubeflow [11:05] Creating standards and rules when creating the platform   [13:29] Snitches [14:10] Inter-team discussions when processes fall apart [16:56] Examples of the development process on the feedback of ML engineers and data scientists [20:35] Preserving flexibility when introducing new models and formats [21:37] Organizational structure of Interos [23:40] Surface area for product [24:46] Use of Git Ops to manage boarding pass [28:04] Cultural emphasis [30:02] Naming conventions [32:28] Benefit of a clean slate [33:16] One-size-fits-all choice [37:34] Wrap up
undefined
Apr 4, 2022 • 51min

Bringing Audio ML Models into Production // Valerio Velardo // MLOps Coffee Sessions #90

MLOps Coffee Sessions #90 with Valerio Velardo, Bringing Audio ML Models into Production. // Abstract The majority of audio/music tech companies that employ ML still don’t use MLOps regularly. In these companies, you rarely find audio ML pipelines which take care of the whole ML lifecycle in a reliable and scalable manner. Audio ML probably pays the price of being a small sub-discipline of ML. It’s dwarfed by ML applications in image processing and NLP. In audio ML, novelties tend to travel slowly. However, things are starting to change. A few audio and music tech companies are investing in MLOps. Building MLOps solutions for music presents unique challenges because audio data is significantly different from all other data types. // Bio Valerio is MLOps Lead at Utopia Music. He’s also an AI audio consultant who helps companies implement their AI music vision by providing technical, strategy, and talent sourcing services. Valerio is interested both in the R&D and productization (MLOps) aspects of AI applied to the audio and music domains. He's the host of The Sound of AI, the largest YouTube channel and online community on AI audio with more than 22K subscribers. Previously, Valerio founded and led Melodrive, a tech startup that developed an AI-powered music engine capable of generating emotion-driven video game music in real-time. Valerio earned a Ph.D. in music AI from the University of Huddersfield (UK). // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Valerio's website https://valeriovelardo.com/ The Sound of AI YouTube channel: https://www.youtube.com/channel/UCZPFjMe1uRSirmSpznqvJfQ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Valerio on LinkedIn: https://www.linkedin.com/in/valeriovelardo/ Timestamps: [00:00] Introduction to Valerio Velardo [01:28] Please subscribe and rate us! [02:40] History of Valerio's love for music [04:12] Intervention of computer science, AI, and Machine Learning in music [08:06] Experimenting with Machine Learning [09:25] Environmental Sound AI [11:05] AI Music [15:22] Traditional ML life cycle within music tech companies [18:02] Representation of data [22:22] Audio being better served in the market [30:42] Success metrics   [35:17] Challenges when talking to R&D teams [38:10] Things need to be battle-hardened before production [39:09] Education process besides Valerio's Youtube channel [42:38] Rectifying use cases not related to audio [45:48] Organizing modular blocks building stacks [47:59] Open-source tools implementation [50:28] Wrap up
undefined
Mar 31, 2022 • 53min

A Journey in Scaling AI // Gabriel Straub // MLOps Coffee Sessions #89

MLOps Coffee Sessions #89 with Gabriel Straub, A Journey in Scaling AI.   // Abstract Gabriel talks to us about the difficulties of scaling ML products across an organization. He speaks about differences in profiles of data consumers and data producers, and the challenges of educating engineers so they have greater insights into the effects that their changes to the system may have. // Bio Gabriel joined Ocado Technology in 2020 as Chief Data Officer, bringing over 10 years of experience in leading data science teams and helping organizations realize the value of their data. At Ocado Technology his role is to help the organization take advantage of data and machine learning so that we can best serve our retail partners and their customers. Gabriel is a guest lecturer at London Business School and an Honorary Senior Research Associate at UCL. He has also advised start-ups and VCs on data and machine learning strategies. Before joining Ocado, Gabriel was previously Head of Data Science at the BBC, Data Director at notonthehighstreet.com, and Head of Data Science at Tesco.    Gabriel has a MA in Mathematics from Cambridge and an MBA from London Business School. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Website: https://www.ocadogroup.com/about-us/ocado-technology Podcast: https://www.reinfer.io/podcast/ai-pioneers-gabriel-straub-chief-data-scientist-ocado Blog: https://www.ocadogroup.com/technology/blog --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Gabriel on LinkedIn: https://www.linkedin.com/in/gabriel-s-65081521/ Timestamps: [00:00] Introduction to Gabriel Straub [03:14] Best of Slack Newsletter [04:06] Gabriel's best purchase since the pandemic [05:37] Ocado's background and Gabriel's role [07:55] Sliding scale of Ocado [10:05] Different use cases of Ocado [12:02] Realizing value with Machine Learning [13:18] How things need to be computed on the edge [14:51] Ocado's main day-to-day [16:17] Being generalizable and when to stop [19:11] The Golden Path [21:30] Foundational level of maturity [24:41] Metrics of success [27:10] Lifespan of a data [28:49] Hard lessons learned from producers and consumers [30:19] Internal assessment [32:50] Evolution of Ocado   [36:58] Rule-based system [38:58] Putting data science and/or machine learning value in front of the consumers [41:55] Going past the constraints [44:24] What holds people back? [46:30] Instilling cultural value of doing right and well into the company [49:42] Being defensive talking about AI [51:44] Ocado is hiring!
undefined
Mar 28, 2022 • 54min

ML Platform Tradeoffs and Wondering Why to Use Them // Javier Mansilla // MLOps Coffee Sessions #88

MLOps Coffee Sessions #88 with Javier Andres Mansilla, ML Platform Tradeoffs and Wondering Why to Use Them. // Abstract Javier runs ML Platform at Mercado Libre. We’re here with Javier because he’s going to tell us about what the ML platform at Mercado Libre looks like granularly, talk about its purpose, lessons, wins, and future improvements, and share with us some of the most challenging use cases they’ve had to engineer around. // Bio During the last 3 years building the internal ML platform for Mercado Libre (NASDAQ MELI), the biggest company in Latam, and the eCommerce & fintech omnipresent solution for the continent. Seasoned entrepreneur and leader, Javier was co-founder and CTO of Machinalis, a hi-end company building Machine Learning since 2010 (yes, before the breakthrough of neuralnets). When Machinalis got acquired by Mercado Libre, that small team evolved to enable Machine Learning as a capability for a tech giant with more 10k devs, impacting the lives of almost 100 million direct users. On a daily basis, Javier leads not only the tech and product roadmap of their Machine Learning Platform, but also their users' tracking system, the AB Testing framework, and the open-source office. Javier loves hanging out with family and friends, python, biking,  football, carpentry, and slow-paced holidays in nature! // MLOps Jobs board   https://mlops.pallet.xyz/jobs  // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Javier on LinkedIn: https://www.linkedin.com/in/javimansilla/ Timestamps: [00:00] Introduction to Javier Andres Mansilla [02:18] Refresher to what Mercado Libre is [06:16] Centralization of Machine Learning platform at Mercado Libre [11:58] Mercado Libre's working size [16:15] Hitting the scale [21:07] Driving ML platform vision and team's business metrics   [28:23] Education process of how to use machine learning on the platform [36:49] Composition of the team members and finding the right people [43:05] Stakeholders [45:32] Decision making [48:51] Wrap up [49:52] Bonus from Javier
undefined
Mar 17, 2022 • 52min

Don't Listen Unless You Are Going to Do ML in Production // Kyle Morris // MLOps Coffee Sessions #87

MLOps Coffee Sessions #87 with Kyle Morris, Don't Listen Unless You Are Going to Do ML in Production. // Abstract Companies wanting to leverage ML specializes in model quality (architecture, training method, dataset), but face the same set of undifferentiated work they need to productionize the model. They must find machines to deploy their model on, set it up behind an API, make the inferences fast, cheap, reliable by optimizing hardware, load-balancing, autoscaling, clustering launches per region, queueing long-running tasks... standardizing docs, billing, logging, CI/CD that integrates testing, and more. Banana.dev's aim is to simplify this process for all. This talk outlines our learnings and the trials and tribulations of ML hosting. // Bio Hey all! Kyle did self-driving AI @ Cruise, robotics @ CMU, currently in business @ Harvard. Now he's building banana.dev to accelerate ML! Kyle cares about safely building superhuman AI. Our generation has the chance to build tools that advance society 100x more in our lifetime than in all of history, but it needs to benefit all living things! This requires a lot of technical + social work. Let's go! // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links kyle.af --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/ Connect with Kyle on LinkedIn: https://www.linkedin.com/in/kylejohnmorris/ Timestamps: [00:00] Introduction to Kyle Morris [02:42] banana.dev   [04:43] banana.dev's vision [06:22] banana.dev's goal beyond the competition [07:28] Computer vision optimization [08:46] Common pitfalls [11:47] Machine Learning Engineering vs Software Engineering [13:47] Who do you hire? [15:12] Disconnect in operationalizing [18:53] Meeting SLOs if stuff is breaking upstream [19:48] Is breaking upstream a part of quality? [21:16] Scenario of what to focus on [24:02] Advice to people dealing with unrealistic expectations [28:11] Hard truth [29:35] MLOps Jobs board - https://mlops.pallet.xyz/jobs [30:42] Don't Listen Unless You Are Going to Do ML in Production [33:15] Hurdle in productionizing ML systems [37:56] Chaos engineering [42:40] War stories [45:54] Catalyst on changing the original post on Kyle's blog [50:11] Wrap up [51:02] Message banana.dev or Kyle if you have questions regarding production. It's free of charge!
undefined
Mar 12, 2022 • 48min

Building ML/Data Platform on Top of Kubernetes // Julien Bisconti // MLOps Coffee Sessions #86

MLOps Coffee Sessions #86 with Julien Bisconti, Building ML/Data Platform on Top of Kubernetes.   // Abstract When building a platform, a good start would be to define the goals and features of that platform, knowing it will evolve. Kubernetes is established as the de facto standard for scalable platforms but it is not a fully-fledged data platform.   Do ML engineers have to learn and use Kubernetes directly?   They probably shouldn't. So it is up to the data engineering team to provide the tools and abstraction necessary to allow ML engineers to do their work.   The time, effort, and knowledge it takes to build a data platform is already quite an achievement. When it is built, one has to maintain it, monitor it, train people to on-call rotation, implement escalation policies and disaster recovery, optimize for usage and costs, secure it and build a whole ecosystem of tools around it (front-end, CLI, dashboards).   That cost might be too high and time-consuming for some companies to consider building their own ML platform as opposed to cloud offering alternatives. Note that cloud offerings still require some of those points but most of the work is already done. // Bio Julien is a software engineer turned Site Reliability Engineer. He is a Google developer expert, certified Data Engineer on Google Cloud and Kubernetes Administrator, mentor for Woman Developer Academy and Google For Startups program. He is working on building and maintaining data/ML platform. // Related Links https://portal.superwise.ai/ Crossing the River by Feeling the Stones • Simon Wardley • GOTO 2018: https://www.youtube.com/watch?v=2IW9L1uNMCs --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Julien on LinkedIn: https://www.linkedin.com/in/julienbisconti/ Timestamps: [00:00] French intro by Julien [00:32] Introduction to Julien Bisconti [03:35] Arriving at the non-technical side process of MLOps [06:06] Envious of people with technological problems [07:27] People problem bandwidth conversation [11:04] Atomic decision making [14:20] Advice to developers either to buy or build in their career potential [18:23] Jobs board - https://mlops.pallet.xyz/jobs [21:28] Chaos engineering [26:33] Role of chaos engineering in building production machine learning systems [32:59] Core challenge of MLOps [37:04] Standardization on an industry level [40:30] Reconciliation of trade-offs using Vertex and Sagemaker [45:21] Crossing the River by Feeling the Stones talk by Simon Wardley   [47:22] Wrap up
undefined
Mar 10, 2022 • 45min

Continuous Deployment of Critical ML Applications // Emmanuel Ameisen // MLOps Coffee Sessions #85

MLOps Coffee Sessions #85 with Emmanuel Ameisen, Continuous Deployment of Critical ML Applications. // Abstract Finding an ML model that solves a business problem can feel like winning the lottery, but it can also be a curse. Once a model is embedded at the core of an application and used by real users, the real work begins. That's when you need to make sure that it works for everyone, that it keeps working every day, and that it can improve as time goes on. Just like building a model is all about data work, keeping a model alive and healthy is all about developing operational excellence. First, you need to monitor your model and its predictions and detect when it is not performing as expected for some types of users. Then, you'll have to devise ways to detect drift, and how quickly your models get stale. Once you know how your model is doing and can detect when it isn't performing, you have to find ways to fix the specific issues you identify. Last but definitely not least, you will now be faced with the task of deploying a new model to replace the old one, without disrupting the day of all the users that depend on it. A lot of the topics covered are active areas of work around the industry and haven't been formalized yet, but they are crucial to making sure your ML work actually delivers value. While there aren't any textbook answers, there is no shortage of lessons to learn. // Bio Emmanuel Ameisen has worked for years as a Data Scientist and ML Engineer. He is currently an ML Engineer at Stripe, where he worked on helping improve model iteration velocity. Previously, he led Insight Data Science's AI program where he oversaw more than a hundred machine learning projects. Before that, he implemented and deployed predictive analytics and machine learning solutions for Local Motion and Zipcar. Emmanuel holds graduate degrees in artificial intelligence, computer engineering, and management from three of France’s top schools. // Related Links https://www.amazon.com/Building-Machine-Learning-Powered-Applications/dp/149204511X https:// www.oreilly.com/library/view/building-machine-learning/9781492045106/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/ Connect with Emmanuel on LinkedIn: https://www.linkedin.com/in/ameisen/ Timestamps: [00:00] Introduction to Emmanuel Ameisen [03:38] Building Machine Learning Powered Applications book inspiration [05:19] The writing process [07:04] Over-engineering NLP [09:13] CV driven development: intentional or natural [11:09] Attribute to machine learning team [14:44] Shortening iteration cycle [16:41] Advice on how to tackle iteration [20:00] Failure modes [21:02] Infrastructure Iteration at Stripe [27:06] Deployment Steps tests challenges [29:34] "You develop operational excellence by exercising it." - Emmanuel Ameisen [33:22] Death of a thousand cuts: Balance of work vs productionization piece balance [36:15] Reproducibility headaches [40:04] Pipelines as software product [41:25] Get the book Building Machine Learning Powered Applications: Going from Idea to Product book by Emmanuel Ameisen! [42:04] Takeaways and wrap up

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode