MLOps podcast #192 with Chris Van Pelt, CISO and co-founder of Weights & Biases, Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines sponsored by  @WeightsBiases . // Abstract Chris, provides insights into his machine learning (ML) journey, emphasizing the significance of ML evaluation processes and the evolving landscape of MLOps. The conversation covers effective evaluation metrics, demo-driven development nuances, and the complexities of ML Ops pipelines. Chris reflects on his experience with Crowdflower, detailing its transition to Weights and Biases and stressing the early integration of security measures. The discussion extends to the transformative impact of ML on the tech industry, challenges in detecting subtle bugs, and the potential of open-source models and multimodal capabilities. // Bio Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://wandb.ai/site ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/ Timestamps: [00:00] Chris' preferred coffee [00:33] Takeaways[03:50] Huge shout out to Weights & Biases for sponsoring this episode! [04:15] Please like, share, and subscribe to our MLOps channels! [04:25] CrowdFlower [07:02] Difference of CrowdFlower and Trajectory [09:13] Transition from CrowdFlower to Weights & Biases [13:05] Excel spreadsheets being passed around via email [15:45] Evolution of Weights & Biases [19:24] CISO role [22:23] Advise for easy wins [25:32] Transition into LLMs [27:36] Prompt injection risks on data [29:42] LLMs for New Personas [34:42] Iterative Value Evaluation Process [36:36] Iterating on New Release [39:31] Evaluation survey [43:21] Landscape of LLMs and its evolution [45:40] Conan O'Brien [46:48] Wrap up

Gregory Kamradt, Founder of @DataIndependent, discusses challenges in implementing AI, transformative potential of AI in business processes, growth of OpenAI. He shares insights into his latest project, a smart companion app analyzing startup pitches. They explore AI experimentation, AI gateway concept, niche models, Google's strategic direction, and OpenAI's copyright protection measures.

Ads Dawson, Senior Security Engineer at Cohere, discusses securing large language models and NLP APIs. Topics include threat modeling, data breaches, defending against attacks, OWASP Top 10 vulnerabilities, Generative AI Red Teaming, and model hallucination. Also explores practical learning, prompt injections, model monitoring, data drift, and LLM top 1010.com project.

In this podcast, Rohit Agarwal, CEO of Portkey.ai, discusses designing for forward compatibility in Gen AI. He explores advancements in generative AI and the importance of a tooling layer. The podcast also covers the evolving landscape of software engineering tools, the ops layer for Janet Away applications, challenges in using open-source models, the technology stack in app production, the importance of an AI gateway, and the challenges of open-sourcing a product.

Anand Das, co-founder and CTO of Bito, discusses the impact of LLMs in tech stack and product development, including their Chrome extension 'explain code'. They explore topics such as optimizing at lower levels, ensuring privacy in non-open source projects with AI language models, and best practices for debugging with coding assistants and learning models.

Faizaan Charania, Product Manager, AI at LinkedIn, discusses his AI product development approach, integrating generative AI, and evaluating AI-driven experiences. He shares wisdom on feedback integration, user trust, and collaboration challenges between product managers and AI teams. The podcast explores push vs pool mechanisms in product development, pros and cons of using open source models in enterprise-level use cases, prompt engineering and iteration, building trust in new interaction patterns, and the shift towards understanding machine learning and natural language models.

Mike Del Balso, CEO of Tecton, and Josh Wills, Angel Investor, discuss the importance of feature stores and platforms for ML applications. They explore challenges in operationalizing data pipelines, integrating offline and online data stores, and templatizing workflows for fraud detection and recommendation systems. The concept of gamification in feature creation is also explored.

Luigi Patruno, VP of Data Science at 2U, Inc, shares insights on managing data products and the challenges that come with it. He discusses effective communication, transitioning from an individual contributor to a data science leader, the importance of discipline and focus in goal-setting, data science leadership and incentives, the value of specific and meaningful positive reinforcement, setting boundaries as a manager, and the importance of solutions and identifying blind spots.

Richa Sachdev, Executive Director- Data Operations and Automation, shares insights on optimizing the end-user experience, understanding business goals and metrics, converting legacy applications, working with data platforms, challenges of integrating different databases, and the role of automation in streamlining processes and improving customer interactions in the reward space.

Ketan Umare, CEO at Union.ai, discusses ML orchestration vs MLOps, prioritizing features for open-source and enterprise projects, challenges of ML orchestration, building a flexible architecture, Unionverse flight and plugin models, challenges of large language models and evaluation.