MLOps Coffee Sessions #151 with Jean-Michel Daignan, ML in Production: A DS from Ubisoft Perspective, co-hosted by Abi Aryan. // Abstract As a data scientist himself, Jean-Michel has a unique perspective on the needs of data scientists when it comes to platform development. He talks about the non-invasive approach his team is taking to bring people onto the platform and their SDK, Merlin. The team is focused on tying machine learning products back to business use cases and the ROI they provide. Abby and Jean-Michel also discuss the use of generative AI and the importance of balancing delivering value and building things quickly. Jean-Michel's blog posts on the topic are recommended for further reading. // Bio The author of the blog "the-odd-dataguy.com" has been a data scientist for over 4.5 years at Ubisoft. Prior to joining the video game industry, Jean-Michel had a background in engineering from France and had previously worked in the energy sector. The blog focuses on topics related to data and machine learning, showcasing the author's expertise in the field. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Blog page: https://www.the-odd-dataguy.com/ Bringing Machine Learning to Production at Ubisoft (PydataMTL June22): https://www.the-odd-dataguy.com/2022/12/29/recap_pydata_mtl_june22/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/ Connect with Jean-Michel on LinkedIn: https://www.linkedin.com/in/jeanmicheldaignan/ Timestamps: [00:00] Jean-Michel's preferred beverage [00:19] Jean-Michel Daignan's background [00:28] Takeaways [04:30] Rate us and share the podcasts with your friends! [05:37] Jean-Michel's projects at Ubisoft [07:48] Jean-Michel's success as a Data Scientist [09:45] Ubisoft basics [10:40] Jean-Michel's success from the downfalls of being a data scientist [12:18] Building for data scientists' considerations [13:57] Differences in designing for data scientists in general [16:35] End twin pipelines and their functions [19:35] Major problems doing maintenance [20:53] Data quality ownership [22:33] Monitoring levels [24:25] Locomotive systems [26:14] Merlin [29:12] DS storage systems [31:09] Feature stores batch or streaming? [32:19] Bringing Machine Learning to Production at Ubisoft blog post [35:10] Features and recommendation systems [37:03] Playing games [38:21] Play data = play personalities [39:42] Deep learning in all the diffusion models or the foundation models [43:06] Servicing data scientists' needs [45:28] Ubisoft's data volume [48:00] Wrap up

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian. // Abstract Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event! // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links LLM in Production survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewform Virtual LLMs in Production Conference registration: https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13 Chinchilla papers: https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/ Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/ Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/ Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/ Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/ Timestamps: [00:00] Round table success to Virtual LLM in Production Conference on April 13th! [00:18] Register for the Virtual LLM in Production Conference now! [00:44] LLM in Production survey [01:40] Lightning round of introduction of speakers [04:34] Large Language Models definition [09:17] What do we consider large? [10:35] Thought process in use cases production [14:30] LLM open source huge movements [16:50] Problems qualifications [19:25] Production use cases frameworks directions [25:25] Open-source language models tokenizer [26:25] Language models democratization [29:25] Three categories for LLMs in Production [31:22] Latency at 2 levels [33:27] Defining production [34:57] Hitting the latency problems [38:20] Fundamental latency barrier [40:39] Latency use case requirement [44:25] Costs and the use cases [48:12] Product management involvement in costing [49:38] LLMs Hallucination definition [52:05] Building deterministic systems trust [55:21] Wrap up

MLOps Coffee Sessions #150 with Saahil Jain, The Future of Search in the Era of Large Language Models, co-hosted by David Aponte. // Abstract Saahil shares insights into the You.com search engine approach, which includes a focus on a user-friendly interface, third-party apps, and the combination of natural language processing and traditional information retrieval techniques. Saahil highlights the importance of product thinking and the trade-offs between relevance, throughput, and latency when working with large language models. Saahil also discusses the intersection of traditional information retrieval and generative models and the trade-offs in the type of outputs they produce. He suggests occupying users' attention during long wait times and the importance of considering how users engage with websites beyond just performance. // Bio Saahil Jain is an engineer at You.com. At You.com, Saahil builds searching and ranking systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. His research work has been published in machine learning conferences such as EMNLP, NeurIPS Datasets & Benchmarks, and ACM-CHIL among others. He has publicly released various machine learning models, methods, and datasets, which have been used by researchers in both academic institutions and hospitals across the world, as part of an open-source movement to democratize AI research in medicine. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science at Columbia University and Stanford University respectively. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: http://saahiljain.me/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/ Connect with Saahil on LinkedIn: https://www.linkedin.com/in/saahiljain/ Timestamps [00:00] Saahil's preferred coffee [04:32] Saahil Jain's background [04:44] Takeaways [07:49] Search Landscape [12:57] Use cases exploration [14:51] Differentiating what to give to users [17:19] Search key challenges [20:05] Search objective relevance [23:22] MLOps Search and Recommender Systems [26:54] Addressing Latency Issues [29:41] Throughput presenting results [32:20] Compute challenges [34:24] Working at a small start-up [36:10] Citations critics [39:17] Use cases to build [40:40] Integrating to Leveraging You.com [42:26] Open AI [46:13] Interfacing with bugs [49:16] Staying focused [52:05] Retrieval augmented models [52:32] Closing thoughts [53:47] Wrap up

MLOps Coffee Sessions #149 with Jason McCampbell, The Challenges of Deploying (many!) ML Models, co-hosted by Abi Aryan and sponsored by Wallaroo. // Abstract In order to scale the number of models a team can manage, we need to automate the most common 90% of deployments to allow ops folks to focus on the challenging 10% and automate the monitoring of running models to reduce the per-model effort for data scientists. The challenging 10% of deployments will often be "edge" cases, whether CDN-style cloud-edge, local servers, or running on connected devices. // Bio Jason McCampbell is the Director of Architecture at Wallaroo.ai and has over 20 years of experience designing and building high-performance and distributed systems. From semiconductor design to simulation, a common thread is that the tools have to be fast, use resources efficiently, and "just work" as critical business applications.    At Wallaroo, Jason is focused on solving the challenges of deploying AI models at scale, both in the data center and at "the edge". He has a degree in computer engineering as well as an MBA and is an alum of multiple early-stage ventures. Living in Austin, Jason enjoys spending time with his wife and two kids and cycling through the Hill Country. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://wallaroo.ai MLOps at the Edge Slack channel: https://mlops-community.slack.com/archives/C02G1BHMJRL --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Jason on LinkedIn: https://www.linkedin.com/in/jasonmccampbell/ Timestamps: [00:00] Jason's preferred coffee [01:22] Takeaways [06:06] MLOps at the Edge Slack channel [06:36] Shoutout to Wallaroo! [07:34] Jason's background [09:54] Combining Edge and ML [11:03] Defining Edge Computing [13:21] Data transport restrictions [15:02] Protecting IP in Edge Computing [17:48] Decentralized Teams and Knowledge Sharing [20:45] Real-time Data Analysis for Improved Store Security and Efficiency [24:49] How to Ensure Statistical Integrity in Federated Networks [26:50] Architecting ML at the Edge [30:44] Machine Learning models adversarial attacks [33:25] Pros and cons of baking models into containers [34:52] Jason's work at Wallaroo [38:22] Navigating the Market Edge [40:49] Last challenges to overcome [44:15] Data Science Use Cases in Logistics [46:27] Vector trade-offs [49:34] AI at the Edge challenges [50:40] Finding the Sweet Spot [53:46] Driving revenue [55:33] Wrap up

Karl Fezer, Intelligence & MLOps expert, discusses biases, defining intelligence, and the future of large language models in AI. He emphasizes the importance of efficient high-impact tasks in MLOps. The conversation touches on philosophical tangents but relates back to practical applications of these concepts.

MLOps Coffee Sessions #147 with Alex DeBrie, Something About Databases co-hosted by Abi Aryan. // Abstract For databases, it feels like we're in the middle of a big shift. The first 10-15 years of the cloud were mostly about using the same core infrastructure patterns but in the cloud (SQL Server, MySQL, Postgres, Redis, Elasticsearch).   In the last few years, we're finally seeing data infrastructure that is truly built for the cloud. Elastic, scalable, resilient, managed, etc. Early examples were Snowflake + DynamoDB. The most recent ones are all the 'NewSQL' contenders (Cockroach, Yugabyte, Spanner) or the 'serverless' ones (Neon, Planetscale). Also seeing improvements in caching, search, etc. Exciting times! // Bio Alex is an AWS Data Hero and self-employed AWS consultant and trainer. He is the author of The DynamoDB Book, a comprehensive guide to data modeling with DynamoDB. Previously, he worked for Stedi and for Serverless, Inc., creators of the Serverless Framework. He loves being involved in the AWS & serverless community, and he lives in Omaha, NE with his family. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Key Takeaways from the DynamoDB Paper: https://www.alexdebrie.com/posts/dynamodb-paper/ Understanding Eventual Consistency in DynamoDB: https://www.alexdebrie.com/posts/dynamodb-eventual-consistency/ Two Scoops of Django 1.11: Best Practices for the Django Web Framework: https://www.amazon.com/Two-Scoops-Django-1-11-Practices/dp/0692915729CAP or no CAP? Understanding when the CAP theorem applies and what it means: https://www.alexdebrie.com/posts/when-does-cap-theorem-apply/ Stop fighting your database/ The DynamoDB book: https://dynamodbbook.com/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Alex on LinkedIn: https://www.linkedin.com/in/alex-debrie/ Timestamps: [00:00] Alex's preferred coffee [00:27] Introduction to Alex DeBrie and DynamoDB [01:05] Takeaways [03:47] Please write down your reviews and you might have a book of Alex! [04:57] Alex's journey from being an Attorney to becoming a Data Engineer [07:31] Why engineering? [10:07] Serverless Data [12:54] Before Airflow [15:41] Batch vs streaming [17:03] Difficulties in Batch and Streaming side [19:45] Modern data infrastructure databases [24:37] Cloud Ecosystem Maturity [27:59] New generation type of Snowflake [29:47] Comparing databases [30:58] What's next on connectors from 2 perspectives? [34:25] Management services at the MLOps level [36:38] DynamoDB [39:32] Why do you like DynamoDB? [41:00] Data used in DynamoDB and size limits [43:46] Comparison of tradeoffs between DynamoDB and Redis [45:52] Preferred opinionated databases [48:43] CAP or no CAP? Understanding when the CAP theorem applies and what it means [52:10] The DynamoDB book [56:17] Chapter you want to expand on the book [57:43] Next book to write [59:25] ChatGPT iterations [1:01:59] Data modeling book wished to be written [1:03:27] Wrap up

MLOps Coffee Sessions #146 with Shalabh Chaudri, The Ops in MLOps - Process and People co-hosted by Abi Aryan. // Abstract Shalabh talks through their newfound appreciation for the MLOps perspective from a customer success standpoint. Shalabh's emphasis on setting realistic expectations and ensuring the delivery of promised value adds is particularly valuable.     Generally, this episode provides a unique and insightful perspective on MLOps from the lens of customer success. // Bio Shalabh has worked in the MLOps domain since 2020 at Algorithmia and Union AI. His experience spans startups and small and large public companies. He has 10+ years of experience in the design, delivery, adoption, and business value realization of B2B infrastructure and platform solutions. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://www.union.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Shalabh on LinkedIn: https://www.linkedin.com/in/shalabhchaudhri/ Timestamps: [00:00] Shalabh's preferred coffee [01:18] Takeaways [02:57] Huge shout out to Union AI! [03:46] Reviews [05:26] Shalab's journey [07:00] The people and process of MLOps [10:25] Accuracy measures and Multiple Stakeholders [13:01] UnionAI's success where others fall short [14:45] Legacy equipment [17:06] Legacy tools versus open source [19:27] Cataloging solution [22:51] Stakeholders and maturity levels [24:26] People and Process in MLOps [29:00] Collaboration for Machine Learning [31:08] Overcoming challenges [34:17] AI and leadership decision-making [35:33] Legacy Companies and AI [39:39] Common pitfalls   [42:24] Neglecting ROI [46:25] Speaking to each level [49:50] Being realistic [51:29] Becoming a champion [53:08] Transitioning to machine learning [55:25] Customer's Skill and Success needed in ML   [57:46] Different sizes of companies 

MLOps Coffee Sessions #145 with Sahil Khanna, Griffin, ML platform at Instacart co-hosted by Mike Del Balso. // Abstract The conversation revolves around the journey of Instacart in implementing machine learning, starting from batch processing to real-time processing. The speaker highlights the importance of real-time processing for businesses and the relevance of Instacart's journey to other machine learning teams.    Sahil emphasizes the soft factors, such as staying customer-focused and the right approach, that contributed to the success of Instacart's machine learning implementation. We also recommend two blog posts by Sahil about Instacart's journey. // Bio Sahil is currently a machine learning engineer at Instacart, where they are building a centralized platform for the training, deployment, and management of diverse ML applications. Before Instacart, Sahil developed ML training and inference platforms at Etsy. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mike on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/ Connect with Sahil on LinkedIn: www.linkedin.com/in/sahil-khanna-umd

MLOps Coffee Sessions #144 with Matthew Dombrowski, Non-traditional Career Paths in MLOps co-hosted by Mihail Eric. // Abstract Let's explore the different aspects of ML and data roles and the variety of responsibilities each role entails! This conversation emphasizes the need for understanding the unique insights each role provides and the similarities in responsibilities and soft skills that are required across different roles.    This episode also highlights the significance of stakeholder alignment in the context of working in big companies and the importance of navigating these complexities for a successful career in ML. // Bio Matt has performed a number of MLOps positions including Solutions Consultant, Solutions Architect, and Product Manager from startups to large organizations. In his current role, Matt builds tools to help social media influencers discover unique and exciting Amazon products to recommend to their audiences. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/ Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewdombrowski/ Timestamps: [00:00] Matt's preferred coffee [00:28] Mihail's new creation [05:09] Introduction to Matthew Dombrowski [06:02] Takeaways [08:30] Pizza and coffee nerds [10:54] Data careers [13:35] Matt's progression through the ml sphere [20:10] Dealing with machine learning [23:20] Transition from deep technical implementer to PM role [27:42] Data is a product [29:30] From start-ups to big companies [32:41] Ambiguity of ML [36:17] Matt's daily routine [40:23] Social media influencers [42:07] Developer advocate [44:00] Stakeholder alignment [49:41] Non-traditional career paths military influence [54:11] Good ways to recommend people to get into ML [57:56] MLOps Meetups all over the world [59:00] Wrap up

MLOps Coffee Sessions #143 with Jill Chase & Manmeet Gujral, Investing in the Next Generation of AI & ML. // Abstract Investors are currently focusing on developer tooling and the foundational AI model movement, as they have seen explosive growth in this area. This podcast explores the impact of foundational models on investment thesis and the future of machine learning operations. The discussion also touches on the idea of generative AI and large language models, and their potential impact on MLOps in the next 10 years. Jill and Manmeet from Capital G share their insights on this topic. // Bio Jill Chase Jill is an investor at CapitalG where she focuses on enterprise software, with an emphasis on data infrastructure and AI/ML. Prior to joining CapitalG, Jill worked in senior startup operating roles, both as the CEO of a private equity-backed business and as the founder of a Y Combinator-backed startup. Jill graduated magna cum laude from Williams College with a dual degree in Economics and Psychology and was captain of the women’s basketball team. She came out to the West Coast to earn an MBA from the Stanford Graduate School of Business, but she was born and raised in Boston where she had the opportunity to cheer on the most impressive era of professional sports a city has ever experienced (Go Patriots). She lives in the Bay Area with her husband where they spend weekends doing as many outside activities as possible, such as pickleball, tennis, hiking, and running. Manmeet Gujral Manmeet is a member of the CapitalG investment team where he focuses on enterprise software, AI & ML, open source, and product-led-growth companies. Prior to joining CapitalG in 2021, Manmeet worked in product marketing and operations at Tecton. Before that, he worked as a consultant at Bain & Company in San Francisco where he specialized in the go-to-market strategy for technology companies and private equity investment diligence. Manmeet is originally from Albany, New York, and graduated from Dartmouth College with a dual degree in Computer Science and Economics. Manmeet is highly opinionated about pizza, an avid New York sports fan, and always willing to share his latest house or hip-hop playlists. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Jill on LinkedIn: https://www.linkedin.com/in/jill-greenberg-chase-53747538/ Connect with Manmeet on LinkedIn: https://www.linkedin.com/in/manmeet-gujral/ Timestamps: [00:00] Manmeet and Jill's preferred coffee [00:25] Takeaways [01:31] CapitalG, Jill and Manmeet's Background [05:12] Sideswiping MLOps by Foundational Models [08:50] MLOps space and the market revenue   [14:50] Foundational models B to C style [20:37] Foundational models taking over [23:00] Uninnovative sentiments [27:50] 2 prototypes of companies [31:51] Finding product market fit [36:20] MLOps market growth changes [40:30] Monster valuations [41:43] The ones that got away [44:07] Wrap up