MLOps.community

MLOps community meetup #53! Last Wednesday we talked to Krishna Gade, CEO & Co-Founder, Fiddler AI. // Abstract: Training and deploying ML models have become relatively fast and cheap, but with the rise of ML use cases, more companies and practitioners face the challenge of building “Responsible AI.” One of the barriers they encounter is increasing transparency across the entire AI lifecycle to not only better understand predictions, but also to find problem drivers. In this session with Krishna Gade, we will discuss how to build AI responsibly, share examples from real-world scenarios and AI leaders across industries, and show how Explainable AI is becoming critical to building Responsible AI. // Bio: Krishna is the co-founder and CEO of Fiddler, an Explainable AI Monitoring company that helps address problems regarding bias, fairness and transparency in AI. Prior to founding Fiddler, Gade led the team that built Facebook’s explainability feature ‘Why am I seeing this?’. He’s an entrepreneur with a technical background with experience creating scalable platforms and expertise in converting data into intelligence. Having held senior engineering leadership roles at Facebook, Pinterest, Twitter, and Microsoft, he’s seen the effects that bias has on AI and machine learning decision-making processes, and with Fiddler, his goal is to enable enterprises across the globe solve this problem. ----------- Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Krishna on LinkedIn: https://www.linkedin.com/in/krishnagade/ Timestamps: [00:00] Thank you Fiddler AI! [01:04] Introduction to Krishna Gade [03:19] Krisha's Background [08:33] Everything was fine when you were doing it behind the scenes. But then when you put it out into the wild, we just lost our "baby." It's no longer under our control. [08:53] "You want to have the assurance of how the system works. Even if it's working fine or if it's not working fine."   [09:37] What else is Explainability? Can you break that down for us? [13:58] "Explainability becomes the cornerstone technology to have in place for you to build Responsible AI in production." [14:48] For those used cases that aren't as high stakes, do you feel it's important? Is it up the foodchain? [18:47] Can we dig into that used case real fast? [22:01] If it is a human doing it, there's a lot more room for error? Bias or theories can be introduced and then they don't have a basis in reality? [23:51] Do you need these subject matter experts or someone who is very advanced to be able to set up what the Explainability tool should be looking for at first is it that plug and play and it will know it latches on to the model? [29:36] Does Explainable AI also entail Explainable Data. I see the point where Explainability can help with getting the insights about data after the model has been trained but should it be handled perhaps more proactively where you unbias the data before training the model on it? [32:16] As a data scientist, there are situations when the prediction output is expected to support a business decision taken by senior executives. In that case, when the Explainable model gives out a prediction that doesn't align with the stakeholder's expectations, how should one navigate through this tricky situation? [43:49] How are denen gram clustering for data explainability?

This is a deep dive into the most recent MLOps Engineering Labs from the point of view of Team 1. // Diagram Link: https://github.com/mlops-labs-team1/engineering.labs#workflow --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Alexey on LinkedIn: https://www.linkedin.com/in/alexeynaiden/ Connect with John on LinkedIn: https://www.linkedin.com/in/johnsavageireland/ Connect with Michel on LinkedIn: https://www.linkedin.com/in/michel-vasconcelos-8273008/ Connect with Varuna on LinkedIn: https://www.linkedin.com/in/vpjayasiri/ Timestamps [00:00] Introduction to Engineering Labs Participants [00:34] What are the Engineering Labs? [01:05] Credits to Ivan Nardini who organized this episode!   [04:24] John Savage Profile [05:13] Did you want to learn MLFlow before this? [05:50] Alexey Naiden Profile   [07:26] Varuna Jayasiri Profile [08:28] Michel Vasconcelos Profile [10:07] Do something with Pytorch and MLFlow and then figure out the rest: What did the process look like for you all?  What have you created? [13:39] What did the implementation look like? How you went about structuring and coding it? [17:03] Did you encounter problems along the way? [20:26] Can you give us a rough overview of what you designed and then where was the first problem you saw? [23:08] Was there a lot to catch up with or did you feel it was fine. Can you explain how it was? [24:12] Talk to us about this tool that you have that John was calling out. What was it called? [24:41] Is this homegrown? You built this? [24:51] Did you guys implement this when you went to the engineering labs? [26:03] Can you take us through the pipeline and then the serving and what the overall view of the diagram is? [37:26] For a pet project it works well, but when you wanna start adding a little bit more on top of it wasn't doing the trick? [38:13] So you see it coming in it's much less of an integral part, another lego building block that is part of the whole thing? [40:54] Did you all have trouble with Pytorch or MLFlow? [42:44] Along with that, what was the prompt you were encountering when you were trying to use Torchserve? [44:27] What are you thinking would have been better in that case? [49:05] Feedback on how Engineering Labs went [50:20] Michel: "Engineering Labs should go on. I would like to be a part of it in the next lab." [51:52] Varuna: "This gives me a tangible thing to look at at any point in time and learn from it." [53:00] John: "I feel I have an anchor into the world of MLOps from having done this lab." [55:52] Alexey: "We're at a checkpoint where there are ways we could take" [56:01] Terraform piece Michel wrote for reproducibility.

MLOps community meetup #52! Last Wednesday we talked to Luke Feeney and Gavin Mendel-Gleason, TerminusDB. // Abstract: A look at the open-source 'Git for Data' landscape with a focus on how the various tools fit into the pipeline. Following that scene-setting, we will delve into how and why TerminusDB builds a revision control database from the ground up. // Takeaways - Understanding the 'git for data' offering and landscape - See how to technically approach a revision control database implementation - Dream of a better tomorrow // Bio: Luke Feeney Operations Lead, TerminusDB   Luke Feeney is Operations Director at TerminusDB. Prior to joining TerminusDB, Luke worked in the Irish Foreign Ministry for a number of years. He served in Ireland’s Permanent Mission to the UN in New York and the Embassies in South Africa and Greece. He was Ireland’s acting Ambassador to Greece for 2016 and 2017. Luke was also the Head of the Government of Ireland’s Brexit Communications Team and the Government Brexit Spokesperson from 2017 to 2018. Gavin Mendel-Gleason Chief Technology Officer, TerminusDB   Dr Gavin Mendel-Gleason is CTO of TerminusDB. He is a former research fellow at Trinity College Dublin in the School of Statistics and Computer Science. His research focuses on databases, logic and verification in software engineering. His work includes contributing to the Seshat global historical databank, an ambitious project to record and analyse patterns in human history. He is the inventor of the Web Object Query Language and the primary architect of TerminusDB. He is interested in improving the best practices of the software development community and a strong believer in formal methods and the use of mathematics and logic as disciplines to increase the quality and robustness of software. ----------- Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Luke on LinkedIn: https://www.linkedin.com/in/luke-feeney/ Connect with Gavin on LinkedIn: https://www.linkedin.com/in/gavinmendelgleason/ Timestamps: [00:00] MLOps Announcements [00:17] Slack Community [00:59] Luke and Gavin's Presentation Style [01:34] MLOps Community Twitter, LinkedIn and Youtube [01:45] Introduction to Luke Feeney and Gavin Mendel-Gleason [04:35] Luke: You wanted Git for Data? [05:17] Deep Breath || Is there a Git for Data? [06:30] What is Git for Data? [08:55] Four Big Buckets [28:43] Jupiter Notebook [30:20] Gavin: Collaboration for Structured Data [31:28] What about gitdifs with gitlfs? [31:40] Outline: Motivation, Challenges, Solution [35:35] Motivation: Why Structured Data? [36:08] Data is Core [37:34] Challenges: Data is Still in the Dark Ages [37:40] Structured or Unstructured, we're doing it wrong [40:15] Managing Data means Collaborating [45:09] Discoverability and Schema: Structured data requires a real database - not just GIT. [46:27] Revision Control [47:00] Collaboration [48:38] "Git for data, data is the new oil." [49:01] Why merging is so difficult? [49:25] "If you have a schema, you can do much more intelligent things." [52:36] Machine Learning and Revision Control

MLOps community meetup #51! Last Wednesday we talked to Pamela Jasper, AI Ethicist, Founder, Jasper Consulting Inc. // Abstract: One of the challenges to the widespread adoption of AI Ethics is not only its integration with MLOps, but the added processes to embed ethical principles will slow and impede Innovation. I will discuss ways in which DS and ML teams can adopt Agile practices for Responsible AI. // Bio: Pamela M. Jasper, PMP is a global financial services technology leader with over 30 years of experience developing front-office capital markets trading and quantitative risk management systems for investment banks and exchanges in NY, Tokyo, London, and Frankfurt. Pamela developed a proprietary Credit Derivative trading system for Deutsche Bank and a quantitative market risk VaR system for Nomura. Pamela is the CEO of Jasper Consulting Inc, a consulting firm through which she provides advisory and audit services for AI Ethics governance. Based on her experience as a software developer, auditor and model risk program manager, Pamela created an AI Ethics governance framework called FAIR – Framework for AI Risk which was presented at the NeurIPS 2020 AI conference. Pamela is available as an Advisor, Auditor and Keynote Speaker on AI Ethics Governance. She is a member of BlackInAI, The Professional Risk Managers Industry Association, Global Association of Risk Managers and ForHumanity. //Takeaways Agile methods of adopting AI Ethical processes. ----------- Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Pamela on LinkedIn: https://www.linkedin.com/in/pamela-michelle-j-a5a3a914/ Timestamps: [00:00] Introduction to Pamela Jasper [00:17] Pamela's Background [05:45] Agile IA/Agile Machine Learning, If they are the right fit for each other? [07:50] What is agile? Not necessarily in and of itself a hard-coded framework. [08:05] Agile itself based on May 2001 Manifesto is simply a set of values and principles and teams that make decisions around these values and principles. [10:17] Proposal of Pamela: Let's do Agile with the underlying Ethics that are involved in the ways that you're creating this machine learning. Is that correct? [10:28] "What I'm suggesting is that Ethics become baked into almost to the mindset of a machine learning engineer, data scientists and in the machine learning operational process for MLOps." [14:37] "Not all models are created equal" [15:59] How would be in an Agile way put into practice in your mind? [36:38] What are the things that would help bridge the gap between AI Ethics and the Agile? [41:01] It's not that you're trying to bring on the Agile framework to the different pieces of Ethics. It's that you're bringing that into the Agile framework? [41:21] "We're weaving Ethics into the bedrock of existing Machine Learning practices." [45:13] How can you really get a diverse team if you're not hiring someone who's there as a diverse person? [48:59] What would Epics look like if you're baking Ethics? [52:52] How do you apply Ethics to an ethically questionable domain like gambling? [54:42] "I think that we can create an AI app for gambling is legal that becomes legal in that construct." [56:23] Do you think it's possible/desirable to automate any of the ethical considerations in this way?

Coffee Sessions #29 with Jet Basrawi of Satalia, Culture and Architecture in MLOps. //Bio Jet started his career in technology as a game designer but became interested in programming. He found he loved it. It was endlessly challenging and deeply enjoyable "Flow" activity. It was also nice to be in demand and earn a living. In the last several years, Jet been passionate about DevOps as a key strategic practice. About a year ago, he came into the AI world and it is a great place to be for someone like him. The challenges of MLOps and all the things surrounding AI delivery is a great space to work in. At about the time Jet got into AI the MLops community began, and it was a great experience to come on the journey with Demetrios who was uncovering topics in parallel to him. It was uncanny that each week Demetrios would run a meetup that dealt with exactly the topics he has been trying to reason about. Jet is very interested in culture and architecture and looking forward to exploring this subject in conversation. //Takeaways Insight into the role of culture and architecture in MLOps. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Jet on LinkedIn: https://www.linkedin.com/in/jet-basrawi-4b9ab43/ Timestamps: [00:00] Introduction to Jet Basrawi [01:24] Jet's take on MLOps [02:00] "MLOps - the real Kung fu's in the future" Jet [02:35] Jet's different opinion on "Tooling is the biggest piece in MLOps".   [04:23] MLOps is a way of life. It's a lifestyle. It's not just tooling. [04:38] Why do you have to move over to the cultural side and where feel things fail culturally when it comes to machine learning? [05:47] What you refer to as an orthodox perspective on DevOps and how that place out in your perspective on MLOps?   [06:37] Why do you believe that the separate terminology is coming about and do you believe that this is ultimately harmful to organizations to have this confusion or do you think things should be simplified? [09:05] As soon as you go down and you're not looking at the big picture. You go down one level and they divert completely, is that your thought too? [12:30] How do you go about educating yourself and then figuring out how to articulate MLOps or constitutes in your organization? [16:16] How to do things differently? What are some of your preferred tactics? How to encourage culture change?   [19:02] "Management is NOT Leadership" [20:13] Why are people stuck in their agile approach? [23:57] Someone's trying to pick something up for the 1st time and then put it into production, how dangerous that can be? [25:53] Accepting failure [29:11] What are some of your principles that helped you communicate to the developers? [35:33] "It has to dumb down." [37:43] Annotation [39:37] "Patterntastic" [41:24] "MLOps is a people problem." [43:50] Sprint are adequate for machine learning? [47:03] "Software development is a social activity" [48:03] "We are all juniors in this field." //Show Notes https://www.youtube.com/watch?v=J1WpAJRt3rg Charlie You https://youtu.be/J36xHc05z-M Manoj https://www.youtube.com/watch?v=vH7UFZZdja8&t=5s Lak design patterns https://www.youtube.com/watch?v=9g4deV1uNZo&t=1s flavios talk https://continuousdelivery.com/implementing/culture/ westrum culture   https://www.youtube.com/watch?v=Y4H8dW7Ium8&feature=youtu.be&t=109 Jez Humble

MLOps community meetup #50! Last Wednesday we talked to Michael Del Balso, Willem Pienaar and David Aronchick, // Abstract: The MLOps tooling landscape is confusing. There’s a complicated patchwork of products and open-source software that each cover some subset of the infrastructure requirements to get ML to production. In this session - we’ll focus on the two most important platforms: model management platforms and feature stores. Model management platforms such as Kubeflow help you get models to production quickly and reliably. Feature stores help you easily build, use, and deploy features. Together, they cover requirements to get models and data to production - the two most important components of any ML project.   In this panel discussion, we’ll be joined by David Aronchick (Co-Founder of Kubeflow), Mike Del Balso (Co-Founder of Tecton) and Willem Pienaar (Creator of Feast). These experts will share their perspective on the challenges of Operational ML and how to build the ideal infrastructure stack for MLOps. They’ll talk about the importance of managing models and data with the same engineering efficiency and best practices that we’ve been applying to application code. They’ll discuss the role of Kubeflow, Feast and Tecton, and share their views on the future of MLOps tooling. // Bio: Michael Del Balso CEO & Co-founder, Tecton   Mike is the co-founder of Tecton, where he is focused on building next-generation data infrastructure for Operational ML. Before Tecton, Mike was the PM lead for the Uber Michelangelo ML platform. He was also a product manager at Google where he managed the core ML systems that power Google’s Search Ads business. Previous to that, he worked on Google Maps. He holds a BSc in Electrical and Computer Engineering summa cum laude from the University of Toronto. Willem Pienaar Co-creator, Feast   Willem is currently a tech lead at Tecton where he leads the development of Feast, an open-source feature store for machine learning. Previously he led the ML platform team at Gojek, the Southeast Asian decacorn, which supports a wide variety of models and handles over 100 million orders every month. His main focus areas are building data and ML platforms, allowing organizations to scale machine learning and drive decision making. In a previous life, Willem founded and sold a networking startup. David Aronchick Program Manager, Azure Innovations   David leads works in the Azure Innovation Office on Machine Learning. This means he spends most of my time helping humans to convince machines to be smarter. He is only moderately successful at this.   Previously, he led product management for Kubernetes on behalf of Google, launched Google Kubernetes Engine, and co-founded the Kubeflow project. He has also worked at Microsoft, Amazon, and Chef and co-founded three startups.   ----------- Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Michael on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/ Connect with Willem on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/ Connect with David on LinkedIn: https://www.linkedin.com/in/aronchick/

MLOps community meetup #49! Last Wednesday we talked to Lak Lakshmanan, Data Analytics and AI Solutions, Google Cloud. // Abstract: Design patterns are formalized best practices to solve common problems when designing a software system. As machine learning moves from being a research discipline to a software one, it is useful to catalogue tried-and-proven methods to help engineers tackle frequently occurring problems that crop up during the ML process. In this talk, I will cover five patterns (Workflow Pipelines, Transform, Multimodal Input, Feature Store, Cascade) that are useful in the context of adding flexibility, resilience and reproducibility to ML in production. For data scientists and ML engineers, these patterns provide a way to apply hard-won knowledge from hundreds of ML experts to your own projects. Anyone designing infrastructure for machine learning will have to be able to provide easy ways for the data engineers, data scientists, and ML engineers to implement these, and other, design patterns. // Bio: Lak is the Director for Data Analytics and AI Solutions on Google Cloud. His team builds software solutions for business problems using Google Cloud's data analytics and machine learning products. He founded Google's Advanced Solutions Lab ML Immersion program and is the author of three O'Reilly books and several Coursera courses. Before Google, Lak was a Director of Data Science at Climate Corporation and a Research Scientist at NOAA. ----------- Connect With Us ✌️-------------    Join our Slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Lak on LinkedIn: https://www.linkedin.com/in/valliappalakshmanan/ Timestamps: [00:00] TWIML Con Debate announcement to be hosted by Demetrios on Friday [00:19] Should data scientists know about Kubernetes? Is it just one machine learning tool to rule them all? Or is it going to be the "best-in-class" tool? [00:35] Strong opinion of Lak about "Should data scientists know about Kubernetes?" [05:50] Lak's background into tech [08:07] Which ones you wrote in the book? Is the airport scenario yours? [09:25] Did you write ML Maturity Level from Google? [12:34] How do you know when to bring on perplexity for the sake of making things easier? [16:06] What are some of the best practices that you've seen being used in tooling?   [20:09] How did you come up with writing the book? [20:59] How did we decide that these are the patterns that we need to put in the book? [24:14] Why did I get the "audacity" to think that this is something that is worth doing? [31:29] What would be in your mind some of the hierarchy of design patterns? [38:05] Are there patterns out there that are yet to be discovered? How do you balance the exploitable vs the explorable ml patterns? [42:08] ModelOps vs MLOps [43:08] Do you feel that a DevOps engineer is better suited to make the transition into becoming a Machine Learning engineer? [46:07] Fundamental Machine Design Patterns vs Software Development Design Patterns [49:23] When you're working with the companies at Google, did you give them a toolchain and a better infrastructure or was there more to it? Did they have to rethink their corporate culture because DevOps is often mistaken as just a pure toolchain?

Coffee Sessions #28 with Charlie You of Workday, Lessons learned from hosting the Machine Learning Engineered podcast //Bio Charlie You is a Machine Learning Engineer at Workday and the host of ML Engineered, a long-form interview podcast aiming to help listeners bring AI out of the lab and into products that people love. He holds a B.S. in Computer Science from Rensselaer Polytechnic Institute and previously worked for AWS AI. Charlie is currently working as a Machine Learning Engineer at Workday. He hosts the ML Engineered podcast, learning from the best practitioners in the world.   Check Charlie's podcast and website here: mlengineered.com https://cyou.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Charlie on LinkedIn: https://linkedin.com/in/charlieyou/ Timestamps: [00:00] Introduction to Charlie You [01:50] Charlie's background on Machine Learning and inspiration to create a podcast [06:20] What's your experience been so far as the machine learning engineer and trying to put models into production and trying to get things out that has business value? [07:08] "I started the podcast because as I started working, I had the tingling that machine learning engineering is harder than most people thought, and like way harder than I personally thought." [08:20] What's an example of that where you target someone in your podcast, you keep that learning and you want an extra meeting the next day and say "Hey, actually I'm starting one of the world's experts on this topics and this is what they said"?    [10:06] In a world of tons of traditional software engineering assets and the process you put in place, how have they adopted what they're doing to the machine learning realm?    [19:00] About your podcast, what are some 2-3 most consistent trends that you've been seeing? [21:08] Instead of splintering so much as machine learning monitoring infrastructure specialist, are you going to departmentalize it in the future? [27:22] Is there such a thing as an MLOps engineer right now? [28:50] "We haven't seen a very vocal, very opinionated project manager in machine learning yet." - Todd Underwood [30:18] "Similarly with tooling, we haven't seen the emergence of the tools that encode those best practices." Charlie [31:42] "The day that you don't have to be a subject matter expert in machine learning to feel confident and deploy machine learning products, is the day that you will see the real product leadership in machine learning." Vishnu [34:12] I'd love to hear your take on some more trends that you've been seeing (Security and Ethics) [34:41] "Data Privacy and Security is always at the top of any consideration for infrastructure." Charlie [35:44] That's driven by legal requirements? How do you solve this problem? [37:27] How do we make sure that if that blows up, you're not left with nothing?   [42:28] In your conversations, have you seen people who goes with cloud provider? [43:25] Enterprises have much different incentives than startups do. [45:48] What are some used cases where companies are needing to service their entire needs? [45:48] What are some used cases where companies are needing to service their entire needs? [49:18] What are some takeaways that you had in terms of how you think about your career, what experiences you want to build as this MLOps based engineering is moving so fast?   [56:08] "Your edge is never in the algorithm"

Coffee Sessions #27 with Noah Gift of Pragmatic AI Labs, Practical MLOps  // A “Gift” from Above This week, Demetrios and Vishnu got to spend time with inimitable Noah Gift. Noah is a data science educator, who teaches at Duke, Northwestern, and many other universities, as well as a technical leader through his company Pragmatic AI Labs and past companies. His bio alone would take up this section of the newsletter, so we invite you to check it out here, as well as the rest of his educational content. Read on for some of our takeaways. // HOW is as important as WHAT In our conversation, Noah eloquently pointed out the numerous challenges of bringing ML into production, and especially for making sure it's used positively. It’s not enough to train great models; it’s important to make sure they impact the world positively as their productionized. How models are used is as important as what the model is. Noah specifically commented on externalities and how’s it incumbent on all MLOps practitioners to understand the externalities created by their models. // Just get certified As an educator, Noah has seen front and center how deficits in ML/DS education at the university level have led to the “cowboy” data scientist that doesn’t fit into an effective technical organizational structure. In his courses, Noah emphasizes getting started with off the shelf models and understanding how existing software systems are engineered before committing to building ML systems. Furthermore, Noah suggested getting certifications as a useful way of upskilling for anyone looking to increase their knowledge base in MLOps, especially by cloud providers. // Tech Stack Risk Finally, as many of you do, we debated the relative merits of the major cloud providers (AWS, Azure, and GCP) with Noah. With his vast experience, Noah made a great point about how adopting extremely new tools can sometimes go wrong. In the past, Noah adopted Erlang as a language used in the development of a product. However, as the language never quite took off (in his experience), it became a struggle to hire the right talent to get things done. Readers, as you go about designing and building the MLOps stack, does any part of the process sound like Noah’s experience with Erlang? Tools or frameworks where downstream adoption may end up fractured? We’d love to hear more! Definitely check out Noah’s podcast with us for more awesome nuggets on MLOps. Thanks to Noah for taking the time!  https://noahgift.com/ Noah Gift Machine Learning, Data Science, Cloud & AI Lecturer His most recent books are: Pragmatic A.I.: An introduction to Cloud-Based Machine Learning (Pearson, 2018) Python for DevOps (O’Reilly, 2020).  Cloud Computing for Data Analysis, 2020 Practical MLOps (O'Reilly, 2021 est.) --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Noah on LinkedIn: https://www.linkedin.com/in/noahgift/ [00:00] Introduction to Noah Gift   [03:28] How we can stay pragmatic when it comes to MLOps? [32:45] The worst excuse that you can give somebody is that "I just do this stuff that's hard, intellectually, but departed, makes it work. That's your job."   [33:34] "In academics, we don't do vocational training, we just teach you theory." "In the Master's Degree, we don't do anything that gets you a job." [46:33] MLOps vs Cloud Provider [51:35] GO vs Erlang

MLOps community meetup #48! Last Wednesday, we talked to Manoj  Agarwal, Software Architect at Salesforce. // Abstract: Serving machine learning models is a scalability challenge at many companies. Most applications require a small number of machine learning models (often < 100) to serve predictions. On the other hand, cloud platforms that support model serving, though they support hundreds of thousands of models, provision separate hardware for different customers. Salesforce has a unique challenge that only very few companies deal with; Salesforce needs to run hundreds of thousands of models sharing the underlying infrastructure for multiple tenants for cost-effectiveness. // Takeaways: This talk explains Salesforce hosts hundreds of thousands of models on a multi-tenant infrastructure to support low-latency predictions. // Bio: Manoj Agarwal is a Software Architect in the Einstein Platform team at Salesforce. Salesforce Einstein was released back in 2016, integrated with all the major Salesforce clouds. Fast forward to today and Einstein is delivering 80+ billion predictions across Sales, Service, Marketing & Commerce Clouds per day. //Relevant Links https://engineering.salesforce.com/flow-scheduling-for-the-einstein-ml-platform-b11ec4f74f97 https://engineering.salesforce.com/ml-lake-building-salesforces-data-platform-for-machine-learning-228c30e21f16 ----------- Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Manoj on LinkedIn: https://www.linkedin.com/in/agarwalmk/ Timestamps: [00:00] Happy birthday Manoj! [00:41] Salesforce blog post about Einstein and ML Infrastructure [02:55] Intro to Serving Large Number of Models with Low Latency [03:34] Manoj' background [04:22] Machine Learning Engineering: 99% engineering + 1% machine learning - Alexey Gregorev on Twitter [04:37] Salesforce Einstein [06:42] Machine Learning: Big Picture [07:05] Feature Engineering [07:30] Model Training [08:53] Model Serving Requirements [13:01] Do you standardize on how models are packaged in order to be served and if so, what standards Salesforce require and enforce from model packaging? [14:29] Support Multiple Frameworks   [16:16] Is it easy to just throw a software library in there? [27:06] Along with that metadata, can you breakdown how that goes?   [28:27] Low Latency [32:30] Model Sharding with Replication [33:58] What would you do to speed up transformation code run before scoring? [35:55] Model Serving Scaling [37:06] Noisy Neighbor: Shuffle Sharding [39:29] If all the Salesforce Models can be categorized into different model type, based on what they provide, what would be some of the big categories be and what's the biggest? [46:27] Retraining of the Model: Does that deal with your team or is that distributed out and your team deals mainly this kind of engineering and then another team deal with more machine learning concepts of it? [50:13] How do you ensure different models created by different teams for data scientists expose the same data in order to be analyzed? [52:08] Are you using Kubernetes or is it another registration engine? [53:03] How is it ensured that different models expose the same information?