MLOps.community

Morgan McGuire and Anish Shah discuss the challenges of productionizing large language models, including cost optimization, latency requirements, trust of output, and debugging. They also mention an upcoming AI in Production Conference on February 22 with informative workshops.

Alexandra Diem, Head of Cloud Analytics & MLOps at Gjensidige, discusses challenges of generative AI in sensitive data environments, specialized chatbots, data governance, enabling teams through MVP development, transitioning analysts into data scientists, and the importance of collaboration. Her journey from academia to being a consultant in Norway is also explored.

Aayush Mudgal, Senior Machine Learning Engineer at Pinterest, discusses the evolution of ads ranking at Pinterest, including transitioning to deep learning-based transformer models. Topics covered include challenges in productionizing large language models, transitioning to deep learning models, incorporating sequential signals, multi-task learning, and transfer learning, scaling machine learning at Pinterest, and the use of transformers in ad rankings and recommendation models.

The podcast discusses the complexities of Language Model evaluation, the use of open-source versus private models, and the urgency of getting models into production. It also explores the challenges of evaluating LLM outcomes and highlights the importance of prompt engineering. Additionally, it emphasizes the need to quickly get ML models into production for identifying bottlenecks and setting up metrics.

Guests Matt Bleifer and Mike Eastham from Tecton discuss the challenges and use cases of Large Language Models and feature platforms in MLOps. They also introduce Tecton's new product RIFT, highlight the importance of choosing the right tool for the job, and delve into the design decisions and challenges of data processing and aggregation in a managed service.

QuantumBlack and McKinsey discuss the trade-offs of buying vs building GenAI solutions, including considerations of black box solutions and transparency. They explore the roles of traditional AI and JAN-AI in messaging channels and the generative nature of AI. The challenges and considerations in using APIs for machine learning models are also discussed.

Jon Cooke, founder of Dataception and creator of the Data Product Pyramid, discusses using specialist small language models and graphs to accelerate data product ecosystems. Topics include deconstructed Encoder/Decoder Transformers, data product management, tech to eliminate data grunt work, and building sophisticated analytics in real-time.

Jake Watson, Principal Data Engineer at The Oakland Group, discusses the challenges and importance of data platforms for ML & AI projects. Topics include data engineering, real-time data updates, data modeling, scalability of ML pipelines, and the role of data platforms in the future.

Yujian is working as a Developer Advocate at Zilliz, where they develop and write tutorials for proof of concepts for large language model applications. They also give talks on vector databases, LLM Apps, semantic search, and tangential spaces.MLOps podcast #206 with Yujian Tang, Developer Advocate at Zilliz, RAG Has Been Oversimplified, brought to us by our Premium Brand Partner, Zilliz// AbstractIn the world of development, Retrieval Augmented Generation (RAG) has often been oversimplified. Despite the industry's push, the practical application of RAG reveals complexities beyond its apparent simplicity. This talk delves into the nuanced challenges and considerations developers encounter when working with RAG, providing a candid exploration of the intricacies often overlooked in the broader narrative.// BioYujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published to conferences including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water.// MLOps Jobs board https://mlops.pallet.xyz/jobs// MLOps Swag/Merchhttps://mlops-community.myshopify.com/// Related LinksWebsite: zilliz.com --------------- ✌️Connect With Us ✌️ -------------Join our Slack community: https://go.mlops.community/slackFollow us on Twitter: @mlopscommunitySign up for the next meetup: https://go.mlops.community/registerCatch all episodes, blogs, newsletters, and more: https://mlops.community/Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/Connect with Yujian on LinkedIn: https://linkedin.com/in/yujiantangTimestamps:[00:00] Yujian's preferred coffee[00:17] Takeaways[02:42] Please like, share, and subscribe to our MLOps channels![02:55] The hero of the LLM space[05:42] Embeddings into Vector databases[09:15] What is large and what is small LLM consensus[10:10] QA Bot behind the scenes[13:59] Fun fact getting more context[17:05] RAGs eliminate the ability of LLMs to hallucinate[18:50] Critical part of the rag stack[19:57] Building citations[20:48] Difference between context and relevance[26:11] Missing prompt tooling[27:46] Similarity search[29:54] RAG Optimization[33:03] Interacting with LLMs and tradeoffs[35:22] RAGs not suited for[39:33] Fashion App [42:43] Multimodel Rags vs LLM RAGs[44:18] Multimodel use cases[46:50] Video citations[47:31] Wrap up

Jonathan Frankle, Chief Scientist at Databricks, discusses the realities and usefulness of AI, including face recognition systems, the 'lottery ticket hypothesis,' and robust decision-making protocols for training models. They also explore Jonathan's move into law, his experience with GPUs, and the revolutionary algorithm called Qstar.