We are having another LLMs in-production Virtual Conference. 50+ speakers combined with in-person activities around the world on June 15 & 16. Sign up free here: https://home.mlops.community/home/events/llm-in-prod-part-ii-2023-06-20 // Abstract This panel discussion is centered around a crucial topic in the tech industry - data privacy and security in the context of large language models and AI systems. The discussion highlights several key themes, such as the significance of trust in AI systems, the potential risks of hallucinations, and the differences between low and high-affordability use cases. The discussion promises to be thought-provoking and informative, shedding light on the latest developments and concerns in the field. We can expect to gain valuable insights into an issue that is becoming increasingly relevant in our digital world. // Bio Diego Oppenheimer Diego Oppenheimer is an entrepreneur, product developer, and investor with an extensive background in all things data. Currently, he is a Partner at Factory a venture fund specializing in AI investments as well as interim head of product at two LLM startups. Previously he was an executive vice president at DataRobot, Founder, and CEO at Algorithmia (acquired by DataRobot), and shipped some of Microsoft’s most used data analysis products including Excel, PowerBI, and SQL Server. Diego is active in AI/ML communities as a founding member and strategic advisor for the AI Infrastructure Alliance and MLops.Community and works with leaders to define ML industry standards and best practices. Diego holds a Bachelor's degree in Information Systems and a Masters degree in Business Intelligence and Data Analytics from Carnegie Mellon University Gevorg Karapetyan Gevorg Karapetyan is the co-founder and CTO of ZERO Systems where he oversees the company's product and technology strategy. He holds a Ph.D. in Computer Science and is the author of multiple publications, including a US Patent. Vin Vashishta C-level Technical Strategy Advisor and Founder of V Squared, one of the first data science consulting firms. Our mission is to provide support and clarity for our clients’ complete data and AI monetization journeys. Over a decade in data science and a quarter century in technology building and leading teams and delivering products with $100M+ in ARR. Saahil Jain Saahil Jain is an engineering manager at You.com. At You.com, Saahil builds search, ranking, and conversational AI systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science at Columbia University and Stanford University respectively. Shreya Rajpal Shreya is the creator of Guardrails AI, an open-source solution designed to establish guardrails for large language models. As a founding engineer at Predibase, she helped build the Applied ML and ML infra teams. Previously, she worked at Apple's Special Projects Group on cross-functional ML, and at Drive.ai building computer vision models.

MLOps Coffee Sessions #157 with Katrina Ni & Aaron Maurer, MLOps Build or Buy, Startup vs. Enterprise? co-hosted by Jake Noble of tecton.ai. This episode is sponsored by tecton.ai - Check out their feature store to get your real-time ML journey started. // Abstract There are a bunch of challenges with building useful machine learning at a B2B software company like Slack, but we've built some cool use cases over the years, particularly around recommendations. One of the key challenges is how to train powerful models while being prudent stewards of our clients' essential business data, and how to do so while respecting the increasingly complex landscape of international data regulation. // Bio Katrina Ni Katrina is a Machine Learning Engineer in Slack ML Services Team where they build ML platforms and integrate ML, e.g. Recommend API, Spam Detection, across product functionalities. Prior to Slack, she is a Software Engineer in Tableau Explain Data Team where they build tools that utilize statistical models and propose possible explanations to help users inspect, uncover, and dig deeper into the viz. Aaron Maurer Aaron is a senior engineering manager in the infra organization at Slack, managing both the machine learning team and the real-time services team. In six years at Slack, most of which Aaron spent as an engineer, He worked on the search ranking, recommendation, spam detection, performance anomaly detection, and many other ML applications. Aaron is also an advisor to Eppo, an experimentation platform. Prior to Slack, Aaroon worked as a data scientist at Airbnb, earned a Masters in statistics at the University of Chicago, and helped develop econometric models projecting the Obamacare rollout at Acumen LLC. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jake on LinkedIn: https://www.linkedin.com/in/jakednoble/ Connect with Katrina on LinkedIn: https://www.linkedin.com/in/katrina-ni-660b2590/ Connect with Aaron on LinkedIn: https://www.linkedin.com/in/aaron-maurer-4003b638/ Timestamps: [00:00] Aaron and Katrina's preferred coffee [00:41] Recommender and System and Jake [02:06] Takeaways [05:38] Introduction to Aaron Maurer & Katrina Ni [06:53] Aaron Maurer & Katrina Ni's Recommend API blog post [08:36] 10-pole machine learning use case and Rex's use case [10:14] Genesis of Slack's recommender system framework [11:47] The Special Sauce [12:58] Speaking the same language [15:23] Use case sources [17:08] Slack's feature engineering [17:52] Main CTR models [18:40] Data privacy [21:33] Slack's recommendations problem [22:09] Fine-tuning the generative models [23:30] Cold start problem [26:02] Underrated [28:24] Baseline [28:55] Cold sore space [30:15] LLMs in Production Conference Part 2 announcement! [32:32] Data scientists transition to ML [33:35] Unicorns do exist! [34:43] Diversity of skill set [36:02] The future of ML [38:34] Model Serving [40:11] MLOps Maturity level [43:06] AWS Analogy [45:05] Primary difficulty [48:07] Wrap up

Sign up for the next LLM in production conference here: https://go.mlops.community/LLMinprod Watch all the talks from the first conference: https://go.mlops.community/llmconfpart1 // Abstract In this panel discussion, the topic of the cost of running large language models (LLMs) is explored, along with potential solutions. The benefits of bringing LLMs in-house, such as latency optimization and greater control, are also discussed. The panelists explore methods such as structured pruning and knowledge distillation for optimizing LLMs. OctoML's platform is mentioned as a tool for the automatic deployment of custom models and for selecting the most appropriate hardware for them. Overall, the discussion provides insights into the challenges of managing LLMs and potential strategies for overcoming them. // Bio Lina Weichbrodt Lina is a pragmatic freelancer and machine learning consultant that likes to solve business problems end-to-end and make machine learning or a simple, fast heuristic work in the real world. In her spare time, Lina likes to exchange with other people on how they can implement best practices in machine learning, talk to her at the Machine Learning Ops Slack: shorturl.at/swxIN. Luis Ceze Luis Ceze is Co-Founder and CEO of OctoML, which enables businesses to seamlessly deploy ML models to production making the most out of the hardware. OctoML is backed by Tiger Global, Addition, Amplify Partners, and Madrona Venture Group. Ceze is the Lazowska Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he has taught for 15 years. Luis co-directs the Systems and Architectures for Machine Learning lab (sampl.ai), which co-authored Apache TVM, a leading open-source ML stack for performance and portability that is used in widely deployed AI applications. Luis is also co-director of the Molecular Information Systems Lab (misl.bio), which led pioneering research in the intersection of computing and biology for IT applications such as DNA data storage. His research has been featured prominently in the media including New York Times, Popular Science, MIT Technology Review, and the Wall Street Journal. Ceze is a Venture Partner at Madrona Venture Group and leads their technical advisory board. Jared Zoneraich Co-Founder of PromptLayer, enabling data-driven prompt engineering. Compulsive builder. Jersey native, with a brief stint in California (UC Berkeley '20) and now residing in NYC. Daniel Campos Hailing from Mexico Daniel started his NLP journey with his BS in CS from RPI. He then worked at Microsoft on Ranking at Bing with LLM(back when they had 2 commas) and helped build out popular datasets like MSMARCO and TREC Deep Learning. While at Microsoft he got his MS in Computational Linguistics from the University of Washington with a focus on Curriculum Learning for Language Models. Most recently, he has been pursuing his Ph.D. at the University of Illinois Urbana Champaign focusing on efficient inference for LLMs and robust dense retrieval. During his Ph.D., he worked for companies like Neural Magic, Walmart, Qualtrics, and Mendel.AI and now works on bringing LLMs to search at Neeva. Mario Kostelac Currently building AI-powered products in Intercom in a small, highly effective team. I roam between practical research and engineering but lean more towards engineering and challenges around running reliable, safe, and predictable ML systems. You can imagine how fun it is in LLM era :). Generally interested in the intersection of product and tech, and building a differentiation by solving hard challenges (technical or non-technical). Software engineer turned into Machine Learning engineer 5 years ago.

MLOps Coffee Sessions #156 with Melissa Barr & Michael Mui, Machine Learning Education at Uber co-hosted by Lina Weichbrodt. // Abstract Melissa and Michael discuss the education program they developed for Uber's machine learning platform service, Michelangelo, during a guest appearance on a podcast. The program teaches employees how to use machine learning both in general and specifically for Uber. The platform team can obtain valuable feedback from users and use it to enhance the platform. The course was designed using engineering principles, making it applicable to other products as well. // Bio Melissa Barr Melissa is a Technical Program Manager for ML & AI at Uber. She is based in New York City. She drives projects across Uber’s ML platform, delivery, and personalization teams. She also built out the first version of the ML Education Program in 2021. Michael Mui Melissa is a Staff Technical Lead Manager on Uber AI's Machine Learning Platform team. He leads the Distributed ML Training team which focuses on building elastic, scalable, and fault-tolerant distributed machine learning libraries and systems used to power machine learning development productivity across Uber. He also co-leads Uber’s internal ML Education initiatives. Outside of Uber, Michael also teaches ML at the Parsons School of Design in NYC as an Adjunct Faculty (mostly for the museum passes!) and guest lectures at the University of California, Berkeley. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.uber.com/blog/ml-education-at-uber-program-design-and-outcomes/https://www.uber.com/blog/ml-education-at-uber/https://www.uber.com/en-PH/blog/ml-education-at-uber/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Melissa on LinkedIn: https://www.linkedin.com/in/melissabarr1/ Connect with Michael on LinkedIn: https://www.linkedin.com/in/michael-c-mui/Connect with Lina on LinkedIn: https://www.linkedin.com/in/lina-weichbrodt-344a066a/ Timestamps: [00:00] Melissa and Michael's preferred coffee [01:51] Takeaways [05:40] Please subscribe to our newsletters and leave reviews on our podcasts! [06:18] Machine learning at Uber education program [07:45] The Uber courses [10:03] Tailoring the Uber education system [12:27] Growing out of the ML-Ed platform efforts [14:14] Expanding the ML Market Size [15:23] Relationship evolution [17:36] Reproducibility best practices [21:46] Learning development timeline [26:29] Courses effectiveness evaluation [29:57] Tracking Progress Challenge [31:25] ML platforms for internal tools [35:07] Impact of ML Education at Uber [39:30] Recommendations to companies who want to start an ML-Ed platform [41:12] Early ML Adoption Program [42:11] Homegrown or home-built platform [42:54] Feature creation to a course [45:24] ML Education at Uber: Frameworks Inspired by Engineering Principles [49:42] The Future of ML Education at Uber [52:28] Unclear ways to spread ML knowledge [54:20] Module for Generative AI and ChatGPT [55:05] Measurement of success [56:39] Wrap up

MLOps Coffee Sessions #155 with Matei Zaharia, The Birth and Growth of Spark: An Open Source Success Story, co-hosted by Vishnu Rachakonda. // Abstract We dive deep into the creation of Spark, with the creator himself - Matei Zaharia Chief technologist at Databricks. This episode also explores the development of Databricks' other open source home run ML Flow and the concept of "lake house ML". As a special treat Matei talked to us about the details of the "DSP" (Demonstrate Search Predict) project, which aims to enable building applications by combining LLMs and other text-returning systems. // About the guest: Matei has the unique advantage of being able to see different perspectives, having worked in both academia and the industry. He listens carefully to people's challenges and excitement about ML and uses this to come up with new ideas. As a member of Databricks, Matei also has the advantage of applying ML to Databricks' own internal practices. He is constantly asking the question "What's a better way to do this?" // Bio Matei Zaharia is an Associate Professor of Computer Science at Stanford and Chief Technologist at Databricks. He started the Apache Spark project during his Ph.D. at UC Berkeley, and co-developed other widely used open-source projects, including MLflow and Delta Lake, at Databricks. At Stanford, he works on distributed systems, NLP, and information retrieval, building programming models that can combine language models and external services to perform complex tasks. Matei’s research work was recognized through the 2014 ACM Doctoral Dissertation Award for the best Ph.D. dissertation in computer science, an NSF CAREER Award, and the US Presidential Early Career Award for Scientists and Engineers (PECASE). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://cs.stanford.edu/~matei/ https://spark.apache.org/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Matei on LinkedIn: https://www.linkedin.com/in/mateizaharia/ Timestamps: [00:00] Matei's preferred coffee [01:45] Takeaways [05:50] Please subscribe to our newsletters, join our Slack, and subscribe to our podcast channels! [06:52] Getting to know Matei as a person [09:10] Spark [14:18] Open and freewheeling cross-pollination [16:35] Actual formation of Spark [20:05] Spark and MLFlow Similarities and Differences [24:24] Concepts in MLFlow [27:34] DJ Khalid of the ML world [30:58] Data Lakehouse [33:35] Stanford's unique culture of the Computer Science Department [36:06] Starting a company [39:30] Unique advice to grad students [41:51] Open source project [44:35] LLMs in the New Revolution [47:57] Type of company to start with [49:56] Emergence of Corporate Research Labs [53:50] LLMs size context [54:44] Companies to respect [57:28] Wrap up

MLOps Coffee Sessions #154 with Waleed Kadous, ML Scalability Challenges, co-hosted by Abi Aryan. // Abstract Dr. Waleed Kadous, Head of Engineering at Anyscale, discusses the challenges of scalability in machine learning and his company's efforts to solve them. The discussion covers the need for large-scale computing power, the importance of attention-based models, and the tension between big and small data. // Bio Dr. Waleed Kadous leads engineering at Anyscale, the company behind the open-source project Ray, the popular scalable AI platform. Prior to Anyscale, Waleed worked at Uber, where he led overall system architecture, evangelized machine learning, and led the Location and Maps teams. He previously worked at Google, where he founded the Android Location and Sensing team, responsible for the "blue dot" as well as ML algorithms underlying products like Google Fit. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: anyscale.com https://www.youtube.com/watch?v=hzW0AKKqew4https://www.anyscale.com/blog/WaleedKadous-why-im-joining-anyscale Ray Summit: https://raysummit.anyscale.com/ Anyscale careers: https://www.anyscale.com/careersLearning Ray O'Reilly book. It's free to anyone interested. https://www.anyscale.com/asset/book-learning-ray-oreilly --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Waleed on LinkedIn: https://www.linkedin.com/in/waleedkadous/ Timestamps: [00:00] Waleed's preferred coffee [00:38] Takeaways [07:37] Waleed's background [13:16] Nvidia investment with Rey [14:00] Deep Learning use cases [17:52] Infrastructure challenges [22:01] MLOps level of maturity [26:42] Scale overloading [29:21] Large Language Models [32:40] Balance between fine-tuning forces prompts engineering [35:51] Deep Learning movement [42:05] Open-source models have enough resources [44:11] Ray [47:59] Value add for any scale from Ray [48:55] "Big data is dead" reconciliation [52:43] Causality in Deep Learning [55:16] AI-assisted Apps [57:59] Ray Summit is coming up in September! [58:49] Anyscale is hiring! [59:25] Wrap up

This exclusive podcast episode covers the key findings from the LLM in-production survey that we conducted over the past month. For all the data to explore yourself use this link https://docs.google.com/spreadsheets/d/13wdBwkX8vZrYKuvF4h2egPh0LYSn2GQSwUaLV4GUNaU/edit?usp=sharing Sign up for our LLM in-production conference happening on April 13th (TODAY) here:https://home.mlops.community/home/events/llms-in-production-conference-2023-04-13

MLOps Coffee Sessions #153 with Rodolfo Núñez, Multilingual Programming and a Project Structure to Enable It, co-hosted by Abi Aryan. // Abstract It's really easy to mix different programming languages inside the same project and use a project template that enables easy collaboration. It's not about what language is better, but rather what language solves the given section of your problem better for you. // Bio Rodo has been working in the "Data Space" for almost 7 years. He was a Senior Data Scientist at Entel (a Chilean telecommunications company) and is now a Senior Machine Learning Engineer at the same company, where I also lead three mini teams dedicated to internal cybersecurity; design/promote continuous training for the entire Analytics team and also the whole company; and ensure the improvement of programming practices and code cleanliness standards. Rodo is currently in charge of helping the team put models into production and define the tools that we will use for it. He specializes in R, but he's language/tool agnostic: you should use the tool that best solves your current problem. Rodo studied Mathematical Engineering and MSc in Applied Mathematics at the University of Chile in addition to General Engineering at the École Centrale Marseille. Rodo really likes to share knowledge (bi-directionally) in whatever he thinks he can contribute. Some things that Rodo like teaching are Data Science, Math, Latin Dances, and whatever he thinks he can give to people. Rodo's other interests are computer games (especially Vermintide and Darktide), board games, and dancing to Latin rhythms. Also, he streams some games and Data Science related topics on Twitch. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.twitch.tv/en_codershttps://www.youtube.com/@en_codershttps://www.twitch.tv/rodonunezhttps://github.com/rodo-nunezhttps://github.com/en-coders-cl --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Rodo on LinkedIn: https://www.linkedin.com/in/rodonunez/ Timestamps: [00:00] Rodo's preferred coffee [00:16] Project structure [00:34] Introduction to Rodolfo Núñez [01:20] Takeaways [04:34] Check out our Meetups, podcasts, newsletters, TikTok, and blog posts! [05:50] Why data scientists should know how to code and code properly [10:32] Becoming a team player [14:02] Cookie cutter project [17:50] Markdown and Quarter over Jupyter notebooks [23:18] Data scientists' templates [30:06] Significance of scripts [33:30] Monolith to Microservices [34:33] Reproducibility [36:37] Entire event processing scripts [40:44] In-House cataloging solution [42:08] Data flows [46:00] Bonus topics! [47:23] Elbow methodology [50:17] Idea behind cross sampling [50:51] Machine Learning and MLOps Security at Entel [58:04] Wrap up

Worlds are colliding! This week we join forces with the hosts of the Practical AI podcast to discuss all things machine learning operations. We talk about how the recent explosion of foundation models and generative models is influencing the world of MLOps, and we discuss related tooling, workflows, perceptions, etc. --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

MLOps Coffee Sessions #152 with Keith Trnka, How A Manager Became a Believer in DevOps for Machine Learning. // Abstract Keith Trnka, a seasoned leader in the technology industry, set foot on the MLOps Podcast in a special episode where he shared insights into his experience leading data teams and machine learning teams, becoming a better software engineer, and overseeing a successful migration from a monolith to microservices in the healthcare sector without any downtime. Keith's background includes directing data science at 98.6, improving language models at swipe and nuance, and completing a Ph.D. thesis in language modeling for assistive technology. His work in these areas has contributed to the development of technology applications for healthcare, including telemedicine visits using natural language processing and machine learning. // Bio Keith has been in the industry for about 11 years. Most recently he was the Director of Data Science at 98point6 where we made telemedicine visits easier for doctors using natural language processing, machine learning, backend engineering, AWS, and frontend engineering. Prior to that, Keith improved the language models used in mobile phone keyboards at Swype and Nuance. And before that, He did his Ph.D. thesis in language modeling for assistive technology. Currently, Keith is traveling, mentoring, and doing a side project on machine translation. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Keith on LinkedIn: https://www.linkedin.com/in/keith-trnka/