MLOps.community

Coffee Sessions #53 with David Aponte, Demetrios Brinkmann, and Vishnu Rachakonda, MLOps Insights. //Abstract MLOps Insights from MLOps community core organizers Demetrios Brinkmann, Vishnu Rachakonda, and David Aponte. In this conversation the guys do a deep dive on testing with respect to MLOps, talk about what they have learned recently around the ML field, and what new things are happening with the MLOps community. //Bio David Aponte David is one of the organizers of the MLOps Community. He is an engineer, teacher, and lifelong student. He loves to build solutions to tough problems and share his learnings with others. He works out of NYC and loves to hike and box for fun. He enjoys meeting new people so feel free to reach out to him! Demetrios Brinkmann At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter. Vishnu Rachakonda Vishnu is the operations lead for the MLOps Community and co-hosts the MLOps Coffee Sessions podcast. He is a machine learning engineer at Tesseract Health, a 4Catalyzer company focused on retinal imaging. In this role, he builds machine learning models for clinical workflow augmentation and diagnostics in on-device and cloud use cases. Since studying bioengineering at Penn, Vishnu has been actively working in the fields of computational biomedicine and MLOps. In his spare time, Vishnu enjoys suspending all logic to watch Indian action movies, playing chess, and writing. Other Links: Continuous Delivery for Machine Learning article by Martin Fowler: https://martinfowler.com/articles/cd4ml.html To Engineer Is Human book by Henry Petroski: https://www.amazon.com/Engineer-Human-Failure-Successful-Design/dp/0679734163 ----------- Connect With Us ✌️-------------    Join our Slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Timestamps: [00:14] Tests and how to do tests in MLOps [09:10] Learning from Vishnu and David's new job [12:42] How will it change? [19:48] Forcing to do the right thing vs allowing to do the wrong thing [21:54] Dealing with Machine Learning Models and Data [25:10] Feature store and monitoring compare page

Coffee Sessions #52 with Dave Bergstein, Vector Similarity Search at Scale. //Abstract Ever wonder how Facebook and Spotify now seem to know you better than your friends? Or why the search feature in some products really “gets” you while in other products it feels stuck in the '90s? The difference is vector search— a method of indexing and searching through large volumes of vector embeddings to find more relevant search results and recommendations. Dave Bergstein, the Director of Product at Pinecone, joins us to describe how vector search is used by companies today, what are the challenges of deploying vector search to production applications, and how teams can overcome those challenges even without the engineering resources of Facebook or Spotify. // Bio Dave Bergstein is Director of Product at Pinecone. Dave previously held senior product roles at Tesseract Health and MathWorks where he was deeply involved with productionalizing AI. Dave holds a Ph.D. in Electrical Engineering from Boston University studying photonics. When not helping customers solve their AI challenges, Dave enjoys walking his dog Zeus and CrossFit. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Dave on LinkedIn: https://www.linkedin.com/company/pinecone-io/mycompany/

Coffee Sessions #51 with Sahbi Chaieb, ML security: Why should you care? //Abstract Sahbi, a senior data scientist at SAS, joined us to discuss the various security challenges in MLOps. We went deep into the research he found describing various threats as part of a recent paper he wrote. We also discussed tooling options for this problem that is emerging from companies like Microsoft and Google. // Bio Sahbi Chaieb is a Senior Data Scientist at SAS, he has been working on designing, implementing, and deploying Machine Learning solutions in various industries for the past 5 years. Sahbi graduated with an Engineering degree from Supélec, France, and holds an MS in Computer Science specialized in Machine Learning from Georgia Tech. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Sahbi on LinkedIn: https://www.linkedin.com/in/sahbichaieb/ Timestamps: [00:00] Introduction to Sahbi Chaieb [01:25] Sahbi's background in tech [02:57] Inspiration of the article [09:40] Why should you care about keeping our model secure? [12:53] Model stealing [14:16] Development practices [17:24] Other tools in the toolbox covered in the article [21:29] Stories/occurrences where data was leaked [24:45] EU Regulations on robustness [26:49] Dangers of federated learning [31:50] Tooling status on model security [33:58] AI Red Teams [36:42] ML Security best practices [38:26] AI + Cyber Security [39:26] Synthetic Data [42:51] Prescription on ML Security in 5-10 years [46:37] Pain points encountered

Coffee Sessions #50 with Alex Chung and Srivathsan Canchi, Creating MLOps Standards. // Abstract With the explosion in tools and opinionated frameworks for machine learning, it's very hard to define standards and best practices for MLOps and ML platforms. Based on their building AWS SageMaker and Intuit's ML Platform respectively, Alex Chung and Srivathsan Canchi talk with Demetrios and Vishnu about their experience navigating "tooling sprawl". They discuss their efforts to solve this problem organizationally with Social Good Technologies and technically with mlctl, the control plane for MLOps. // Bio Alex Chung Alex is a former Senior Product Manager at AWS Sagemaker and an ML Data Strategy and Ops lead at Facebook. He's passionate about the interoperability of MLOps tooling for enterprises as an avenue to accelerate the industry. Srivathsan Canchi Srivathsan leads the machine learning platform engineering team at Intuit. The ML platform includes real-time distributed featurization, scoring, and feedback loops. He has a breadth of experience building high scale mission-critical platforms. Srivathsan also has extensive experience with K8s at Intuit and previously at eBay, where his team was responsible for building a PaaS on top of K8s and OpenStack. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Alex on LinkedIn: https://linkedin.com/in/alex-chung-gsd Connect with Sri on LinkedIn: https://www.linkedin.com/in/srivathsancanchi/ Timestamps: [00:00] Introduction to Alex Chung and Srivathsan Canchi [01:36] Alex's background in tech [03:07] Srivathsan's background in tech [04:36] What is SGT? [05:53] 3 Categories of SGT            1. Education            2. Standardization            3. Orchestration   [07:00] Standardization is desirable [13:03] Perspective from both sides   [13:39] Profile breakdown of Standardization [17:20] Importance of Standardization in enterprise [21:02] Tooling sprawl [24:04] Standardizing the different interfaces between MLOps tools [31:54] mlctl [33:35] mlctl's future [38:38] How mlctl helps the workflow of Intuit [41:00] CIGS evolve the different spaces

Coffee Sessions #49 with Stefan Krawczyk, Aggressively Helpful Platform Teams. //Abstract At Stitch Fix there are 130+ “Full Stack Data Scientists” who in addition to doing data science work, are also expected to engineer and own data pipelines for their production models. One data science team, the Forecasting, Estimation, and Demand team were in a bind. Their data generation process was causing them iteration & operational frustrations in delivering time-series forecasts for the business. the solution? Hamilton, a novel python micro-framework, solved their pain points by changing their working paradigm. Some of the main workers on Hamilton are the dedicated engineering team called Data Platform. Data Platform builds services, tools, and abstractions to enable DS to operate in a full-stack manner avoiding hand-off. In the beginning, this meant DS built the web apps to serve model predictions, now as the layers of abstractions have been built over time, they still dictate what is deployed, but write much less code. // Bio Stefan loves the stimulus of working at the intersection of design, engineering, and data. He grew up in New Zealand, speaks Polish, and spent formative years at Stanford, LinkedIn, Nextdoor & Idibon. Outside of work in pre-covid times Stefan liked to 🏊, 🌮, 🍺, and ✈. // Other Links https://www.youtube.com/watch?v=B5Zp_30Knoo https://www.slideshare.net/StefanKrawczyk/hamilton-a-micro-framework-for-creating-dataframes https://www.slideshare.net/StefanKrawczyk/deployment-for-free-removing-the-need-to-write-model-deployment-code-at-stitch-fix --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Stefan on LinkedIn: https://linkedin.com/in/skrawczyk Timestamps: [00:00] Introduction to Stefan Krawczyk [00:37] Why Hamilton? [01:50] Stefan's background in tech [04:15] Model Life Cycle Team [06:48] Managing outcomes generated by data scientists   [09:04] Teams doing the same thing [12:41] Vision of getting code down to zero [18:40] Freedom and autonomy went wrong [21:17] Sub teams   [24:00] Create and deploy models easily [24:28] Interesting challenge to define [25:15] Stitch Fix Model productionization to be proud of [26:23] Hamilton to open-source [28:45] Model Envelope [31:45] Deployment for free [34:53] Use of Model Envelope in Model Artifact [37:16] Extending API definition in a model envelope for the model [39:00] Dependencies [40:08] Monitoring at scale [43:43] Advice in terms of neat abstraction [46:19] Envelope vs Container [47:33] Time frame of Hamilton's development and its benefits

Coffee Sessions #48 with Julien Chaumond, Tour of Upcoming Features on the Hugging Face Model Hub. //Abstract Julien Chaumond’s Tour of Upcoming Features on the Hugging Face Model Hub. Our MLOps community guest in this episode is Julien Chaumond the CTO of Hugging Face - every data scientist’s favorite NLP Swiss army knife. Julien, David, and Demetrios spoke about many topics including: Infra for hosting models/model hubs Inference widgets for companies with CPUs & GPUs (for companies) Auto NLP which trains models “Infrastructure as a service” // Bio Julien Chaumond is Chief Technical Officer at Hugging Face, a Brooklyn and Paris-based startup working on Machine learning and Natural Language Processing, and is passionate about democratizing state-of-the-art AI/ML for everyone. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Julien on LinkedIn: https://www.linkedin.com/in/julienchaumond/ Timestamps: [00:00] Introduction to Julien Chaumond [01:57] Julien's background in tech [04:35] "I have this vision of building a community where the greatest people in AI can come together and basically invent the future of Machine Learning together." [04:55] What is Hugging Face? [06:17] "We have the goal of bridging the gap between research and production on actual production use cases." [06:45] Start of open-source in Hugging Face [07:50] Chatbox experiment (reference resolution system) - linking pronouns to the subjects of sentences [10:20] From a project to a company [11:46] "The goal was to explore in the beginning." [11:57] Importance of platform [14:25] "Transfer learning is an efficient way of Machine Learning. Providing your platform  around change that people want to start from pre-trained model and fine-tune them into the specific use case is something that can be big so we built some stuff to help people do that." [15:35] Narrowing down the scope of service to provide [16:27] "We have some vision of what we want to build but a lot of it is the small incremental improvements that we bring to the platform. I think it's the natural way of building stuff nowadays because Machine Learning is moving so fast." [20:00] Model Hubs [22:37] "We're guaranteeing that we don't build anything that introduces any lagging to Hugging Face because we're using Github. You'll have that peace of mind." [26:31] Storing model artifacts [27:00] AWS - cache - stored to an edge location all around the globe [28:39] Inference widgets powering [27:17] "For each model on the model hub we try to ensure that we have the metadata about the model to be able to actually run it." [32:11] Deploying infra function [32:38] "Depending on the model and library, we optimize the custom containers to make sure that they run as fast as possible on the target hardware that we have."    [34:59] "Machine Learning is still pretty much hardware dependent." [36:11] Hardware usage [39:04] "CPU is super cheap. If you are able to run Berks served with a 1-millisecond on CPU because you have powerful optimizations, you don't really need GPUs anymore. It's cost-efficient and energy-efficient."   [40:30] Challenges of Hugging Face and what you learned [41:10] "It may sound like a super cliche but the team that you assembled is everything." [43:22] War stories in Hugging Face [44:12] "Our goal is more forward-looking to be helpful as much as we can to the community." [48:25] Hugging Face accessibility

Coffee Sessions #47 with Jeremy Howard, fast.ai, AutoML, Software Engineering for ML. //Abstract Advancement in ML Workflows: You've been around the ML world for long enough to have seen how much workflows, tooling, frameworks, etc. have matured and allowed for greater scale and access. We'd love to reflect on your personal journey in this regard and hear about your early experiences putting models into production, as well as how you appreciate/might improve the process now. Data Professional Diversity and MLOps: Your work at fast.ai, Kaggle, and now with NBDEV has played a huge part in supercharging a diverse ecosystem of professionals that contribute to ML-like ML/data scientists, researchers, and ML engineers. As the attention turns to putting models into production, how do you think this range of professionals will evolve and work together? How will things around building models change as we build more? Turning Research into Practice: You've consistently been a leader in applying cutting-edge ideas from academia into practical code others can use. It's one of the things I appreciate most about the fast.ai course and package. How do you go about picking which ideas to invest in? What advice would you give to industry practitioners charged with a similar task at their company? // Bio Jeremy Howard is a data scientist, researcher, developer, educator, and entrepreneur. Jeremy is a founding researcher at fast.ai, a research institute dedicated to making deep learning more accessible. He is also a Distinguished Research Scientist at the University of San Francisco, the chair of WAMRI, and is Chief Scientist at platform.ai. Previously, Jeremy was the founding CEO of Enlitic, which was the first company to apply deep learning to medicine, and was selected as one of the world’s top 50 smartest companies by MIT Tech Review two years running. He was the President and Chief Scientist of the data science platform Kaggle, where he was the top-ranked participant in international machine learning competitions for 2 years running. He was the founding CEO of two successful Australian startups (FastMail, and Optimal Decisions Group–purchased by Lexis-Nexis). Before that, he spent 8 years in management consulting, at McKinsey & Co, and at AT Kearney. Jeremy has invested in, mentored, and advised many startups, and contributed to many open-source projects. He has many media appearances, including writing for the Guardian, USA Today, and The Washington Post, appearing on ABC (Good Morning America), MSNBC (Joy Reid), CNN, Fox News, BBC, and was a regular guest on Australia’s highest-rated breakfast news program. His talk on TED.com, “The wonderful and terrifying implications of computers that can learn”, has over 2.5 million views. He is a co-founder of the global Masks4All movement. // Other Links: jhoward.fastmail.fm enlitic.com jphoward.wordpress.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Jeremy on LinkedIn: https://www.linkedin.com/in/howardjeremy/ Timestamps: [00:00] Introduction to Jeremy Howard [02:11] Jeremy's background [03:10] Workflow [12:59] Platform development [19:53] Balancing API [22:57] Moment of inefficiency [27:42] Helpful tactics [29:05] University of tools evolving   [41:10] Resources to solve problems [43:30] Jupiter notebooks [47:20] Putting Jupiter notebooks into production [48:42] MBDev [51:20] Jeremy's experiences and frustrations with putting ML into production

Coffee Sessions #46 with Pablo Estevez, What We Learned from 150 Successful ML-enabled Products at Booking.com. //Abstract While most of the Machine Learning literature focuses on the algorithmic or mathematical aspects of the field, not much has been published about how Machine Learning can deliver meaningful impact in an industrial environment where commercial gains are paramount. We conducted an analysis on about 150 successful customer-facing applications of Machine Learning, developed by dozens of teams in Booking.com, exposed to hundreds of millions of users worldwide, and validated through rigorous Randomized Controlled Trials. Our main conclusion is that an iterative, hypothesis-driven process, integrated with other disciplines was fundamental to build 150 successful products enabled by Machine Learning. // Bio Pablo Estevez is the Principal Data Scientist at Booking.com. He has worked on recommendations, personalization, and experimentation across the Booking.com website, as well as as a manager on several machine learning, data science, and product development teams. // Other Links Talk on the topic: https://www.youtube.com/watch?v=ljhtfrtuNqw&t=4h24m30s The paper: https://blog.kevinhu.me/2021/04/25/25-Paper-Reading-Booking.com-Experiences/bernardi2019.pdf --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Pablo on LinkedIn: https://www.linkedin.com/in/estevezpablo/

MLOps community meetup #70! Last Wednesday, we talked to Monika Venckauskaite, Senior Machine Learning Engineer at Vinted. //Abstract One of the areas, that is the most transformed by ML these years is cybersecurity. Traditionally, SIEM (Security Intelligence and Event Management) is performed by human analysts. However, as the cyber powers and tools of the world are growing, we need more and more of these specialists. The entire area of cybersecurity is experiencing a shortage of talent. This is where the ML is coming in to help us. Cybersecurity ML systems require a lot of expertise from specialists as well as unique ways of handling user-sensitive data. This imposes various architectural solutions. In this talk, Monika introduces us to the ways of using ML in cybersecurity and the unique challenges we face. //Bio Monika is a keen and curious ML engineer, loving to build systems. She's started in machine learning as a master's student, looking for Higgs Boson and Dark matter within the CERN data. Later on, Monika moved to the IT industry and worked on various machine learning projects, including Open Source Intelligence Tools and a distributed system for ML cybersecurity analytics. Currently, Monika works as an MLOps engineer, improving the MLOps platform that is used in production to shipping models to a 45 million-user platform. Monika also works in a start-up that is innovating satellite communication. In her free time, she loves books, traveling, and playing some music. // Takeaways Cyber threats are all around us. ML as technology is both a savior and a threat. GDPR and sensitive user data bring in extra challenges for cybersecurity intelligence systems, leading to more complex architectural decisions. ML helps to fight the talent shortage. Cybersecurity requires real-time ML systems and reacting ASAP. ----------- Connect With Us ✌️-------------    Join our Slack community:  https://go.mlops.community/slack Follow us on Twitter:  @mlopscommunity Sign up for the next meetup:  https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Monika on LinkedIn: https://www.linkedin.com/in/monika-in-space/ Timestamps: [00:00] Introduction to Monika Venčkauskaitė [05:50] Monika's background in tech [08:50] Machine Learning in Cyber Security [09:37] Content [10:19] Our world is run by machines     [11:16] Cybersecurity Threats [12:44] Cybersecurity Incident Response                         Cycle:                         1. Identify           2. Protect           3. Detect           4. Respond           5. Recover [25:05] The Iceberg                          Surface Web - 4% Indexed and easily searchable            Deep Web - 90% Not Indexed, tougher to find            Dark Web - 6% Obscured, difficult to discover [47:45] Recommendation: AI Superpowers: China, Silicon Valley, And The New World Order by Kai-Fu Lee (https://www.amazon.com/AI-Superpowers-China-Silicon-Valley/dp/132854639X) [50:54] "I think we are going in the same direction but our implementations are different."

Coffee Sessions #45 with Diego Oppenheimer of Algorithmia, Enterprise Security and Governance MLOps. //Abstract MLOps in the enterprise is difficult due to security and compliance. In this MLOps Coffee Session, the CEO of Algorithmia, Diego talks to us about how we can better approach MLOps within the enterprise. This is an introduction to essential principles of security in MLOps and why it is crucial to be aware of security best practices as an ML professional. // Bio Diego Oppenheimer is co-founder and CEO of Algorithmia. Previously, he designed, managed, and shipped some of Microsoft’s most used data analysis products including Excel, Power Pivot, SQL Server, and Power BI. He holds a Bachelor’s degree in Information Systems and a Master’s degree in Business Intelligence and Data Analytics from Carnegie Mellon University. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/ Timestamps: [00:00] Thank you Diego and Algorithmia for sponsoring this session! [01:04] Introduction to Diego Oppenheimer [02:55] Security [04:42] "The level of scrutiny for apps and development and that of the operational software is much higher." [07:40] "We take the Ops part of MLOps very, very seriously and it's really about the operational side of the equation." [09:22] MLSecOps [11:42] "The code doesn't change, but things change cause the data changed." [15:23] Maturity of security [18:45] "To a certain degree, we have general parameters of software DevOps In software engineering and DevOps, and we're adapting it to this new world of ML."   [19:03] Development workflow [20:58] "In the ideal world, you're just sitting in your data science platform, your auto ML platform, whatever it is that you're working with, you can push a model." [22:50] Security, responsibility and authentication [23:38] "What you don't want to learn is how to do automation every single time there's a new use case. That's just not a good use of your time."  [24:30] Hurdles needed to be cleared [24:47] "I would argue that there's no such thing as Bulletproof in software. That doesn't exist. It never has and never will." [26:25] Machine Learning security risks                         1. Operational risk           2. Brand risk           3. Strategic risk [28:23] Machine Learning security risk standards [31:11] "There's a world where you can reverse engineer a model by essentially feeding a whole bunch of data and understanding where that comes back." [33:55] How to change the mindset of relaxed companies when it comes to security [35:19] "It takes time and money to figure out security." [37:52] Conscientious when building systems [39:44] "Look at the end result of the workflow and understand the value of that workflow, which you should know at that point because if you're going into an ML workflow without understanding what the end value is going to be, it's not a good sign." [40:19] Root cause analysis [41:00] Threat modeling [41:14] "There's a natural next step where there's threat modeling for ML systems and it's a task that gets built and understood, and nobody's going to enjoy doing it."   [43:07] Security as code [45:29] MLRE