Contributor cover image

Contributor

LLM Evaluation: Opik with Gideon Mendels

Jan 15, 2025
Gideon Mendels, co-founder and CEO of Comet, dives into the fascinating world of AI with a focus on Opik, his open-source model evaluation platform. He shares how Opik's rise exceeded expectations and emphasizes the critical role of CI/CD in AI development. Gideon also discusses the alarming decline in dedicated machine learning engineers and differentiates between genuine and 'fake' open-source solutions. The conversation wraps up with insights on the evolving AI landscape and the need for organizations to adapt to new evaluation methodologies.
37:59

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

  • Opik emerged as a vital tool for evaluating LLM applications, addressing new developer needs in AI through open-source contributions.
  • The podcast emphasizes the shift towards community-driven innovation in AI, prioritizing evaluation and testing to ensure model performance and reliability.

Deep dives

The Dominance of Open Source in DevTools

Open source software has become the predominant choice in many categories of development tools, particularly in the machine learning and artificial intelligence sectors. The podcast emphasizes that while some proprietary software companies do well, the vast majority of valuable tools derive from open source contributions. This trend is evident in the rise of platforms such as OPIC, which was developed after recognizing the unique needs of developers as they transitioned from training models to utilizing large language models (LLMs) through APIs. As open source continues to demonstrate its effectiveness, it suggests an ongoing shift towards community-driven innovation in the tech industry.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode