Gideon Mendels, CEO and co-founder of Comet, dives into the intricate world of testing and evaluating LLMs. He discusses the hybrid approach required for these applications, merging machine learning with software engineering best practices. Topics include innovative methods for evaluating LLMs beyond traditional metrics, the challenge of unit testing with deterministic assertions, and the importance of experiment tracking in ensuring reproducibility. Gideon also highlights the role of user interaction analysis in enhancing LLM applications' performance.

Gideon Mendels, co-founder and CEO of Comet, dives into the fascinating world of AI with a focus on Opik, his open-source model evaluation platform. He shares how Opik's rise exceeded expectations and emphasizes the critical role of CI/CD in AI development. Gideon also discusses the alarming decline in dedicated machine learning engineers and differentiates between genuine and 'fake' open-source solutions. The conversation wraps up with insights on the evolving AI landscape and the need for organizations to adapt to new evaluation methodologies.

Gideon Mendels, Co-founder and CEO of Comet ML, discusses the intersection of machine learning and software engineering. Topics include model performance evaluation, monitoring ML models in production, basics of machine learning for software engineers, and building an effective machine learning team.