MLOps.community

MLOps Coffee Sessions #155 with Matei Zaharia, The Birth and Growth of Spark: An Open Source Success Story, co-hosted by Vishnu Rachakonda. // Abstract We dive deep into the creation of Spark, with the creator himself - Matei Zaharia Chief technologist at Databricks. This episode also explores the development of Databricks' other open source home run ML Flow and the concept of "lake house ML". As a special treat Matei talked to us about the details of the "DSP" (Demonstrate Search Predict) project, which aims to enable building applications by combining LLMs and other text-returning systems. // About the guest: Matei has the unique advantage of being able to see different perspectives, having worked in both academia and the industry. He listens carefully to people's challenges and excitement about ML and uses this to come up with new ideas. As a member of Databricks, Matei also has the advantage of applying ML to Databricks' own internal practices. He is constantly asking the question "What's a better way to do this?" // Bio Matei Zaharia is an Associate Professor of Computer Science at Stanford and Chief Technologist at Databricks. He started the Apache Spark project during his Ph.D. at UC Berkeley, and co-developed other widely used open-source projects, including MLflow and Delta Lake, at Databricks. At Stanford, he works on distributed systems, NLP, and information retrieval, building programming models that can combine language models and external services to perform complex tasks. Matei’s research work was recognized through the 2014 ACM Doctoral Dissertation Award for the best Ph.D. dissertation in computer science, an NSF CAREER Award, and the US Presidential Early Career Award for Scientists and Engineers (PECASE). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://cs.stanford.edu/~matei/ https://spark.apache.org/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Matei on LinkedIn: https://www.linkedin.com/in/mateizaharia/ Timestamps: [00:00] Matei's preferred coffee [01:45] Takeaways [05:50] Please subscribe to our newsletters, join our Slack, and subscribe to our podcast channels! [06:52] Getting to know Matei as a person [09:10] Spark [14:18] Open and freewheeling cross-pollination [16:35] Actual formation of Spark [20:05] Spark and MLFlow Similarities and Differences [24:24] Concepts in MLFlow [27:34] DJ Khalid of the ML world [30:58] Data Lakehouse [33:35] Stanford's unique culture of the Computer Science Department [36:06] Starting a company [39:30] Unique advice to grad students [41:51] Open source project [44:35] LLMs in the New Revolution [47:57] Type of company to start with [49:56] Emergence of Corporate Research Labs [53:50] LLMs size context [54:44] Companies to respect [57:28] Wrap up

MLOps Coffee Sessions #154 with Waleed Kadous, ML Scalability Challenges, co-hosted by Abi Aryan. // Abstract Dr. Waleed Kadous, Head of Engineering at Anyscale, discusses the challenges of scalability in machine learning and his company's efforts to solve them. The discussion covers the need for large-scale computing power, the importance of attention-based models, and the tension between big and small data. // Bio Dr. Waleed Kadous leads engineering at Anyscale, the company behind the open-source project Ray, the popular scalable AI platform. Prior to Anyscale, Waleed worked at Uber, where he led overall system architecture, evangelized machine learning, and led the Location and Maps teams. He previously worked at Google, where he founded the Android Location and Sensing team, responsible for the "blue dot" as well as ML algorithms underlying products like Google Fit. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: anyscale.com https://www.youtube.com/watch?v=hzW0AKKqew4https://www.anyscale.com/blog/WaleedKadous-why-im-joining-anyscale Ray Summit: https://raysummit.anyscale.com/ Anyscale careers: https://www.anyscale.com/careersLearning Ray O'Reilly book. It's free to anyone interested. https://www.anyscale.com/asset/book-learning-ray-oreilly --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Waleed on LinkedIn: https://www.linkedin.com/in/waleedkadous/ Timestamps: [00:00] Waleed's preferred coffee [00:38] Takeaways [07:37] Waleed's background [13:16] Nvidia investment with Rey [14:00] Deep Learning use cases [17:52] Infrastructure challenges [22:01] MLOps level of maturity [26:42] Scale overloading [29:21] Large Language Models [32:40] Balance between fine-tuning forces prompts engineering [35:51] Deep Learning movement [42:05] Open-source models have enough resources [44:11] Ray [47:59] Value add for any scale from Ray [48:55] "Big data is dead" reconciliation [52:43] Causality in Deep Learning [55:16] AI-assisted Apps [57:59] Ray Summit is coming up in September! [58:49] Anyscale is hiring! [59:25] Wrap up

This exclusive podcast episode covers the key findings from the LLM in-production survey that we conducted over the past month. For all the data to explore yourself use this link https://docs.google.com/spreadsheets/d/13wdBwkX8vZrYKuvF4h2egPh0LYSn2GQSwUaLV4GUNaU/edit?usp=sharing Sign up for our LLM in-production conference happening on April 13th (TODAY) here:https://home.mlops.community/home/events/llms-in-production-conference-2023-04-13

MLOps Coffee Sessions #153 with Rodolfo Núñez, Multilingual Programming and a Project Structure to Enable It, co-hosted by Abi Aryan. // Abstract It's really easy to mix different programming languages inside the same project and use a project template that enables easy collaboration. It's not about what language is better, but rather what language solves the given section of your problem better for you. // Bio Rodo has been working in the "Data Space" for almost 7 years. He was a Senior Data Scientist at Entel (a Chilean telecommunications company) and is now a Senior Machine Learning Engineer at the same company, where I also lead three mini teams dedicated to internal cybersecurity; design/promote continuous training for the entire Analytics team and also the whole company; and ensure the improvement of programming practices and code cleanliness standards. Rodo is currently in charge of helping the team put models into production and define the tools that we will use for it. He specializes in R, but he's language/tool agnostic: you should use the tool that best solves your current problem. Rodo studied Mathematical Engineering and MSc in Applied Mathematics at the University of Chile in addition to General Engineering at the École Centrale Marseille. Rodo really likes to share knowledge (bi-directionally) in whatever he thinks he can contribute. Some things that Rodo like teaching are Data Science, Math, Latin Dances, and whatever he thinks he can give to people. Rodo's other interests are computer games (especially Vermintide and Darktide), board games, and dancing to Latin rhythms. Also, he streams some games and Data Science related topics on Twitch. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.twitch.tv/en_codershttps://www.youtube.com/@en_codershttps://www.twitch.tv/rodonunezhttps://github.com/rodo-nunezhttps://github.com/en-coders-cl --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Rodo on LinkedIn: https://www.linkedin.com/in/rodonunez/ Timestamps: [00:00] Rodo's preferred coffee [00:16] Project structure [00:34] Introduction to Rodolfo Núñez [01:20] Takeaways [04:34] Check out our Meetups, podcasts, newsletters, TikTok, and blog posts! [05:50] Why data scientists should know how to code and code properly [10:32] Becoming a team player [14:02] Cookie cutter project [17:50] Markdown and Quarter over Jupyter notebooks [23:18] Data scientists' templates [30:06] Significance of scripts [33:30] Monolith to Microservices [34:33] Reproducibility [36:37] Entire event processing scripts [40:44] In-House cataloging solution [42:08] Data flows [46:00] Bonus topics! [47:23] Elbow methodology [50:17] Idea behind cross sampling [50:51] Machine Learning and MLOps Security at Entel [58:04] Wrap up

Worlds are colliding! This week we join forces with the hosts of the Practical AI podcast to discuss all things machine learning operations. We talk about how the recent explosion of foundation models and generative models is influencing the world of MLOps, and we discuss related tooling, workflows, perceptions, etc. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

MLOps Coffee Sessions #152 with Keith Trnka, How A Manager Became a Believer in DevOps for Machine Learning. // Abstract Keith Trnka, a seasoned leader in the technology industry, set foot on the MLOps Podcast in a special episode where he shared insights into his experience leading data teams and machine learning teams, becoming a better software engineer, and overseeing a successful migration from a monolith to microservices in the healthcare sector without any downtime. Keith's background includes directing data science at 98.6, improving language models at swipe and nuance, and completing a Ph.D. thesis in language modeling for assistive technology. His work in these areas has contributed to the development of technology applications for healthcare, including telemedicine visits using natural language processing and machine learning. // Bio Keith has been in the industry for about 11 years. Most recently he was the Director of Data Science at 98point6 where we made telemedicine visits easier for doctors using natural language processing, machine learning, backend engineering, AWS, and frontend engineering. Prior to that, Keith improved the language models used in mobile phone keyboards at Swype and Nuance. And before that, He did his Ph.D. thesis in language modeling for assistive technology. Currently, Keith is traveling, mentoring, and doing a side project on machine translation. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Keith on LinkedIn: https://www.linkedin.com/in/keith-trnka/

MLOps Coffee Sessions #151 with Jean-Michel Daignan, ML in Production: A DS from Ubisoft Perspective, co-hosted by Abi Aryan. // Abstract As a data scientist himself, Jean-Michel has a unique perspective on the needs of data scientists when it comes to platform development. He talks about the non-invasive approach his team is taking to bring people onto the platform and their SDK, Merlin. The team is focused on tying machine learning products back to business use cases and the ROI they provide. Abby and Jean-Michel also discuss the use of generative AI and the importance of balancing delivering value and building things quickly. Jean-Michel's blog posts on the topic are recommended for further reading. // Bio The author of the blog "the-odd-dataguy.com" has been a data scientist for over 4.5 years at Ubisoft. Prior to joining the video game industry, Jean-Michel had a background in engineering from France and had previously worked in the energy sector. The blog focuses on topics related to data and machine learning, showcasing the author's expertise in the field. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Blog page: https://www.the-odd-dataguy.com/ Bringing Machine Learning to Production at Ubisoft (PydataMTL June22): https://www.the-odd-dataguy.com/2022/12/29/recap_pydata_mtl_june22/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Jean-Michel on LinkedIn: https://www.linkedin.com/in/jeanmicheldaignan/ Timestamps: [00:00] Jean-Michel's preferred beverage [00:19] Jean-Michel Daignan's background [00:28] Takeaways [04:30] Rate us and share the podcasts with your friends! [05:37] Jean-Michel's projects at Ubisoft [07:48] Jean-Michel's success as a Data Scientist [09:45] Ubisoft basics [10:40] Jean-Michel's success from the downfalls of being a data scientist [12:18] Building for data scientists' considerations [13:57] Differences in designing for data scientists in general [16:35] End twin pipelines and their functions [19:35] Major problems doing maintenance [20:53] Data quality ownership [22:33] Monitoring levels [24:25] Locomotive systems [26:14] Merlin [29:12] DS storage systems [31:09] Feature stores batch or streaming? [32:19] Bringing Machine Learning to Production at Ubisoft blog post [35:10] Features and recommendation systems [37:03] Playing games [38:21] Play data = play personalities [39:42] Deep learning in all the diffusion models or the foundation models [43:06] Servicing data scientists' needs [45:28] Ubisoft's data volume [48:00] Wrap up

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian. // Abstract Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event! // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links LLM in Production survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewform Virtual LLMs in Production Conference registration: https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13 Chinchilla papers: https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/ Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/ Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/ Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/ Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/ Timestamps: [00:00] Round table success to Virtual LLM in Production Conference on April 13th! [00:18] Register for the Virtual LLM in Production Conference now! [00:44] LLM in Production survey [01:40] Lightning round of introduction of speakers [04:34] Large Language Models definition [09:17] What do we consider large? [10:35] Thought process in use cases production [14:30] LLM open source huge movements [16:50] Problems qualifications [19:25] Production use cases frameworks directions [25:25] Open-source language models tokenizer [26:25] Language models democratization [29:25] Three categories for LLMs in Production [31:22] Latency at 2 levels [33:27] Defining production [34:57] Hitting the latency problems [38:20] Fundamental latency barrier [40:39] Latency use case requirement [44:25] Costs and the use cases [48:12] Product management involvement in costing [49:38] LLMs Hallucination definition [52:05] Building deterministic systems trust [55:21] Wrap up

MLOps Coffee Sessions #150 with Saahil Jain, The Future of Search in the Era of Large Language Models, co-hosted by David Aponte. // Abstract Saahil shares insights into the You.com search engine approach, which includes a focus on a user-friendly interface, third-party apps, and the combination of natural language processing and traditional information retrieval techniques. Saahil highlights the importance of product thinking and the trade-offs between relevance, throughput, and latency when working with large language models. Saahil also discusses the intersection of traditional information retrieval and generative models and the trade-offs in the type of outputs they produce. He suggests occupying users' attention during long wait times and the importance of considering how users engage with websites beyond just performance. // Bio Saahil Jain is an engineer at You.com. At You.com, Saahil builds searching and ranking systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. His research work has been published in machine learning conferences such as EMNLP, NeurIPS Datasets & Benchmarks, and ACM-CHIL among others. He has publicly released various machine learning models, methods, and datasets, which have been used by researchers in both academic institutions and hospitals across the world, as part of an open-source movement to democratize AI research in medicine. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science at Columbia University and Stanford University respectively. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: http://saahiljain.me/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Saahil on LinkedIn: https://www.linkedin.com/in/saahiljain/ Timestamps [00:00] Saahil's preferred coffee [04:32] Saahil Jain's background [04:44] Takeaways [07:49] Search Landscape [12:57] Use cases exploration [14:51] Differentiating what to give to users [17:19] Search key challenges [20:05] Search objective relevance [23:22] MLOps Search and Recommender Systems [26:54] Addressing Latency Issues [29:41] Throughput presenting results [32:20] Compute challenges [34:24] Working at a small start-up [36:10] Citations critics [39:17] Use cases to build [40:40] Integrating to Leveraging You.com [42:26] Open AI [46:13] Interfacing with bugs [49:16] Staying focused [52:05] Retrieval augmented models [52:32] Closing thoughts [53:47] Wrap up

MLOps Coffee Sessions #149 with Jason McCampbell, The Challenges of Deploying (many!) ML Models, co-hosted by Abi Aryan and sponsored by Wallaroo. // Abstract In order to scale the number of models a team can manage, we need to automate the most common 90% of deployments to allow ops folks to focus on the challenging 10% and automate the monitoring of running models to reduce the per-model effort for data scientists. The challenging 10% of deployments will often be "edge" cases, whether CDN-style cloud-edge, local servers, or running on connected devices. // Bio Jason McCampbell is the Director of Architecture at Wallaroo.ai and has over 20 years of experience designing and building high-performance and distributed systems. From semiconductor design to simulation, a common thread is that the tools have to be fast, use resources efficiently, and "just work" as critical business applications.    At Wallaroo, Jason is focused on solving the challenges of deploying AI models at scale, both in the data center and at "the edge". He has a degree in computer engineering as well as an MBA and is an alum of multiple early-stage ventures. Living in Austin, Jason enjoys spending time with his wife and two kids and cycling through the Hill Country. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://wallaroo.ai MLOps at the Edge Slack channel: https://mlops-community.slack.com/archives/C02G1BHMJRL --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Jason on LinkedIn: https://www.linkedin.com/in/jasonmccampbell/ Timestamps: [00:00] Jason's preferred coffee [01:22] Takeaways [06:06] MLOps at the Edge Slack channel [06:36] Shoutout to Wallaroo! [07:34] Jason's background [09:54] Combining Edge and ML [11:03] Defining Edge Computing [13:21] Data transport restrictions [15:02] Protecting IP in Edge Computing [17:48] Decentralized Teams and Knowledge Sharing [20:45] Real-time Data Analysis for Improved Store Security and Efficiency [24:49] How to Ensure Statistical Integrity in Federated Networks [26:50] Architecting ML at the Edge [30:44] Machine Learning models adversarial attacks [33:25] Pros and cons of baking models into containers [34:52] Jason's work at Wallaroo [38:22] Navigating the Market Edge [40:49] Last challenges to overcome [44:15] Data Science Use Cases in Logistics [46:27] Vector trade-offs [49:34] AI at the Edge challenges [50:40] Finding the Sweet Spot [53:46] Driving revenue [55:33] Wrap up