MLOps Coffee Sessions #84 with Ernest Chan, Lessons from Studying FAANG ML Systems. // Abstract Large tech companies invest in ML platforms to accelerate their ML efforts. Become better prepared to solve your own MLOps problems by learning from their technology and design decisions. Tune in to learn about ML platform components, capabilities, and design considerations. // Bio Ernest is a Data Scientist at Duo Security. As part of the core team that built Duo's first ML-powered product, Duo Trust Monitor, he faced many (frustrating) MLOps problems first-hand. That led him to advocate for an ML infrastructure team to make it easier to deliver ML products at Duo. Prior to Duo, Ernest worked at an EdTech company, building data science products for higher-ed. Ernest is passionate about MLOps and using ML for social good. // Related Links   Lessons on ML Platforms — from Netflix, DoorDash, Spotify, and more: https://ernestklchan.medium.com/lessons-on-ml-platforms-from-netflix-doordash-spotify-and-more-f455400115c7     Paper Highlights-Challenges in Deploying Machine Learning: a Survey of Case Studies https://towardsdatascience.com/paper-highlights-challenges-in-deploying-machine-learning-a-survey-of-case-studies-cafe61cfd04c     Choose boring technologies Slideshare by Dan McKinley: https://www.slideshare.net/danmckinley/choose-boring-technology --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Ernest on LinkedIn: https://www.linkedin.com/in/ernest-chan-68245773/ Timestamps: [00:00] Introduction to Ernest Chan [01:07] Takeaways [02:58] Ernest's Lessons on ML Platforms — from Netflix, DoorDash, Spotify, and more blog post [05:55] Five components of an ML Platform   [10:09] Limitations highlighted in the blog post [14:41] Level of maturity or completion observed in company efforts [16:17] Platform/Architecture admired the most [17:46] Advice to big tech companies [22:03] Process of needing an infrastructure and aiming towards having a platform [24:23] Paper Highlights-Challenges in Deploying Machine Learning: a Survey of Case Studies blog post [26:24] Takeaways from Paper Highlights-Challenges in Deploying Machine Learning [30:33] Prioritization [33:04] Delta Lake [35:27] Model rollouts and shadow mode [39:23] Are you an ML Engineer or a Data Scientist?   [40:15] Simple route platform vs flexible platform trade-offs [41:08] Opinionated and simple vs less opinionated and flexible [43:22] Choose boring technologies Slideshare by Dan McKinley [44:36] Wrap up

MLOps Coffee Sessions #83 with Vincent Warmerdam, Better Use cases for Text Embeddings. // Abstract Text embeddings are very popular, but there are plenty of reasons to be concerned about their applications. There's algorithmic fairness, compute requirements as well as issues with datasets that they're typically trained on. In this session, Vincent gives an overview of some of these properties while also talking about an underappreciated use-case for the embeddings: labeling! // Bio Vincent D. Warmerdam is a senior data professional who worked as an engineer, researcher, team lead, and educator in the past. He's especially interested in understanding algorithmic systems so that one may prevent failure. As such, he has a preference for simpler solutions that scale, as opposed to the latest and greatest from the hype cycle. He currently works as a Research Advocate at Rasa where he collaborates with the research team to explain and understand conversational systems better. Outside of Rasa, Vincent is also well known for his open-source projects (scikit-lego, human-learn, doubtlab, and more), collaborations with open source projects like spaCy, his blog over at koaning.io, and his calm code educational project. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Skylar on LinkedIn: https://www.linkedin.com/in/skylar-payne-766a1988/ Connect with Vincent on LinkedIn: https://www.linkedin.com/in/vincentwarmerdam/

MLOps Reading Group meeting on February 11, 2022   Reading Group Session about Feature Stores with Matt Delacour and Mike Moran   --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Connect with us on LinkedIn: https://www.linkedin.com/company/mlopscommunity/ Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, Feature Store, Machine Learning Monitoring and Blogs: https://mlops.community/ Timestamps: [00:05] Matt's intro [00:26] Mike's intro [01:09] Matt’s talk: Feature store system at Shopify [01:45] What is Shopify? [02:05] Shopify Use Case [02:38] Choosing a solution [03:19] Managed service vs In-house vs Open-source (Feast) [06:01] Why did we choose Feast? [11:25] Implementation Strategy (multi-repo vs mono-repo approaches) [13:01] Mono-repo approach breakdown [14:30] Internal SDK [17:01] Q&A: Does feast satisfy scalability for online inference of Shopify latency requirements? [19:05] Q&A: Do you rely on Feast to serialize data to the online store? [20:13] Q&A: Is your mono-repo library a subset of Feast? [21:18] Q&A: Did you consider using git submodules for a multi-repo? [23:02] Q&A: Are you storing embeddings with Feast? [24:30] Q&A: Regarding the mono-repo, which modules are responsible for feature engineering? How do you guarantee that different feature engineering can be used across many DS? [27:58] Mike’s talk (Feature store at Skyscanner) [28:08] Kaleidoscope System [28:25] Background and context of the Feature store [29:30] Initial state of the feature store [30:13] How does the marketing team also leverage the feature store [31:04] Current state of the feature store (marketing & machine learning) [31:44] SDK approach of creating schemas with dataframes (easy access) [32:16] Reusability across teams among marketing and DS team [33:06] GDPR constraints [33:34] Data updates at the feature store [36:09] Q&A: When a DS updates a feature, how are you communicating that across teams? [38:25] Q&A: Are you applying different levels of feature engineering to increase the likelihood of a DS going back to a previous checkpoint of processing? [40:55] Q&A: In what languages are you implementing the feature store? [44:28] Q&A: Regarding performance-wise, how do you decide what code remains in Apache Spark vs SQL? [49:00] Wrap-up

MLOps Community Meetup #93! Two weeks ago, we talked to Chad Sanderson, Trustworthy Data for Machine Learning. //Abstract The most common challenge for ML teams operating at scale is data quality. In this talk, Chad discusses how Convoy invested in a large-scale data quality effort to treat data as an API and provide a data change management surface to enable trustworthy machine learning. // Bio Chad Sanderson is the Product Lead for Convoy's Data Platform team, which includes the data warehouse, streaming, BI & visualization, experimentation, machine learning, and data discovery. Chad has built everything from feature stores, experimentation platforms, metrics layers, streaming platforms, analytics tools, data discovery systems, and workflow development platforms. He’s implemented open source, SaaS products (early and late-stage) and has built cutting-edge technology from the ground up. Chad loves the data space, and if you're interested in chatting about it with him, don't hesitate to reach out. // Related links    ----------- ✌️Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Catch all episodes, Feature Store, Machine Learning Monitoring and Blogs: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Chad on LinkedIn: https://www.linkedin.com/in/chad-sanderson/

MLOps Coffee Sessions #82 with Donna Schut and Christos Aniftos, Practitioners Guide to MLOps. // Abstract The "Practitioners Guide to MLOps" introduced excellent frameworks for how to think about the field. Can we talk about how you've seen the advice in that guide applied to real-world systems? Is there additional advice you'd add to that paper based on what you've seen since its publication and with new tools being introduced? Your article about selecting the right capabilities has a lot of great advice. It would be fun to walk through a hypothetical company case and talk about how to apply that advice in a real-world setting. GCP has had a lot of new offerings lately, including Vertex AI. It would be great to talk through what's new and what's coming down the line. Our audience always loves hearing how tool providers like GCP think about the problems customers face and how tools are correspondingly developed. // Bio Donna Schut Donna is a Solutions Manager at Google Cloud, responsible for designing, building, and bringing to market smart analytics and AI solutions globally. She is passionate about pushing the boundaries of our thinking with new technologies and creating solutions that have a positive impact. Previously, she was a Technical Account Manager, overseeing the delivery of large-scale ML projects, and part of the AI Practice, developing tools, processes, and solutions for successful ML adoption. She managed and co-authored Google Cloud’s AI Adoption Framework and Practitioners' Guide to MLOps. Christos Aniftos Christos is a machine learning engineer with a focus on the end-to-end ML ecosystem. On a typical day, Christos helps Google customers productionize their ML workloads using Google Cloud products and services with special attention on scalable and maintainable ML environments. Christos made his ML debut in 2010 while working at DigitalMR, where he led a team of data scientists and developers to build a social media monitoring & analytics tool for the Market Research sector. // Related links:   Select the Right MLOps Capabilities for Your ML Usecase   https://cloud.google.com/blog/products/ai-machine-learning/select-the-right-mlops-capabilities-for-your-ml-use-case Practitioner's Guide to MLOps white paper https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Donna on LinkedIn: https://www.linkedin.com/in/donna-schut/ Connect with Christos on LinkedIn: https://www.linkedin.com/in/aniftos/ Timestamps: [00:00] Introduction to Donna Schut and Christos Aniftos [05:52] Inspiration of Practitioner's Guide to MLOps paper [06:57] Model for working with customers [08:14] Where are we at MLOps? [10:20] Process of working with customers [11:30] Overview of processes and capabilities outlined in Practitioner's Guide to MLOps paper [16:16] Continuous Training maturity levels [22:37] Context about the discovery process [25:21] Disciplines and security mix tend to see [26:12] Is there a level up in maturity? [29:50] Success or failures that stand out [38:00] War stories   [43:16] Internal study of qualities of the best ML engineers

MLOps Coffee Sessions #81 with Davis Treybig and Leigh Marie Braswell, Machine Learning from the Viewpoint of Investors. // Abstract Machine learning is a rapidly evolving space that can be hard to keep track of. Every year, thousands of research papers are published in the space, and hundreds of new companies are built both in applied machine learning as well as in machine learning tooling. In this podcast, we interview two investors who focus heavily on machine learning to get their take on the state of the machine learning industry today: Leigh-Marie Braswell at Founders Fund and Davis Treybig at Innovation Endeavors. We discuss their perspectives on opportunities within MLOps and applied machine learning, common pitfalls and challenges seen in machine learning startups, and new projects they find exciting and interesting in the space. // Bio Davis Treybig Davis (email: davis@innovationendeavors.com) is currently a principal on the investment team at Innovation Endeavors, an early-stage venture firm focused on highly technical companies. He primarily focuses on software infrastructure, especially data tooling and security. Prior to Innovation Endeavors, Davis was a product manager at Google, where he worked on the Pixel phone and the developer platform for the Google Assistant. Davis studied computer science and electrical engineering in college. Leigh Marie Braswell Leigh Marie (Twitter: @LM_Braswell) is an investor at Founders Fund. Before joining Founders Fund, she was an early engineer & the first product manager at Scale AI, where she originally built & later led product development for the LiDAR/3D annotation products, used by many autonomous vehicles, robots, and AR/VR companies as a core step in their machine learning lifecycles. She also has done software development at Blend, machine learning at Google, and quantitative trading at Jane Street. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Leigh on LinkedIn: https://www.linkedin.com/in/leigh-marie-braswell/ Connect with Davis on LinkedIn: https://www.linkedin.com/in/davistreybig/

MLOps Coffee Sessions #80 with Ale Solano, The Journey from Data Scientist to MLOps Engineer. // Abstract After years of failed POCs then all of a sudden one of our models is accepted and will be used in production. The next morning we are part of the main scrum stand-up meeting and a DevOps guy is assisting us. A strange feeling, unknown to us until then, starts growing on the AI team: we are useful! Deploying models to production is challenging, but MLOps is more than that. MLOps is about making an AI team useful and iterative from the beginning. And it requires a role that takes care of the technical challenges that this implies, given the experimental nature of the ML field, while also serving the product and business needs. If your AI team does not include this role, maybe it's your time to step up and do it yourself! Today, we will chat with Ale about the transition from being a data scientist to a self-called MLOps engineer. And yes, you'll need to study computer science. // Bio Ale is born and raised in a mid-small town near Malaga in southern Spain. Ale did his bachelor's degree in robotics because it sounded cool and then he got into machine learning because it was even cooler. Ale worked in two companies as an ML developer. Now he's on a temporary hiatus to study business and computer science and get a motivation boost. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/ Connect with Ale on LinkedIn: https://www.linkedin.com/in/alesolano/

MLOps Coffee Sessions #79 with Orr Shilon, Platform Thinking: A Lemonade Case Study.   // Abstract This episode is the epitome of why people listen to our podcast. It’s a complete discussion of the technical, organizational, and cultural challenges of building a high-velocity, machine learning platform that impacts core business outcomes.    Orr tells us about the focus on automation and platform thinking that’s uniquely allowed Lemonade’s engineers to make long-term investments that have paid off in terms of efficiency. He tells us the crazy story of how the entire data science team of 20+ people was supported by only 2 ML engineers at one point, demonstrating the leverage their technical strategy has given engineers.   // Bio Orr is an ML Engineering Team Lead at Lemonade, currently working an ML Platform, empowering Data Scientists to manage the ML lifecycle from research to development and monitoring.   Previously, Orr worked at Twiggle on semantic search, at Varonis on data governance, and at Intel. He holds a B.Sc. in Computer Science and Psychology from Tel Aviv University.   Orr also enjoys trail running and sometimes races competitively.   --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Orr on LinkedIn: https://www.linkedin.com/in/orrshilon/

MLOps Coffee Sessions #78 with Erica Greene and Seoyoon Park, Calibration for ML at Etsy - apply() special. // Abstract This is a special conversation about Machine Learning calibration at Etsy. Demetrios sat down with Erica Greene and Seoyoon Park to hear about how they implemented Calibration into the Etsy Machine Learning workflow. The conversation is a pre-chat with these two before their presentation at the apply() conference on February 10th. Register here: applyconf.com // Bio Erica Geen Erica is an engineering manager with a background in machine learning. She's passionate about developing programs and policies that support women and other underrepresented groups in technology. Seoyoon Park Backend software engineer and aspiring software architect interested in producing scalable, performant, and fault-tolerant applications by keeping up to date with best practices and industry standards. Seoyoon strives to better himself and his peers by advocating for frequent knowledge transfers and promoting a culture of continuous learning. Constantly looking for opportunities to grow as a developer and become a leader of the industry. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Erica on LinkedIn: https://www.linkedin.com/in/ericagreene/ Connect with Seoyoon on LinkedIn: https://www.linkedin.com/in/seoyoonpark/

MLOps Coffee Sessions #77 with Scott Hirleman, Data Mesh - The Data Quality Control Mechanism for MLOps? // Abstract Scott covers what is a data mesh at a high level for those not familiar. Data mesh is potentially a great win for ML/MLOps as there is very clear guidance on creating useful, clean, well-documented/described and interoperable data for "unexpected use". So instead of data spelunking being a harrowing task, it can be a very fruitful one. And that one data set that was so awesome? Well, it wasn't a one-off, it's managed as a product with regular refreshes! And there is a LOT more ownership/responsibility on data producers to make sure the downstream doesn't break. Might sound like kumbaya for MLOps (or total BS?) re far cleaner data and fewer upstream breaks so let's discuss the realities and limitations! // Bio A self-professed "chaotic (mostly) good character", Scott is focused on helping the data mesh community accelerate towards finding solutions for some of data management's hardest challenges. He founded the Data Mesh Learning community specifically to gather enough people to exchange ideas - much of which is patterned off the MLOps community. He hosts the Data Mesh Radio podcast, where he dives deep into topics related to data mesh to provide the data community with useful perspectives and thoughts on data mesh. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletter and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/ Connect with Scott on LinkedIn: https://www.linkedin.com/in/scotthirleman/