Contributor cover image

Contributor

LLM Evaluation: Opik with Gideon Mendels

Jan 15, 2025
37:59

Gideon Mendels (Github: @gidim) is the co-founder and CEO of Comet, the end-to-end model evaluation platform for AI developers. Among the tools in the Comet ecosystem is Opik, an open-source solution for evaluating, testing and monitoring LLM applications. Opik allows users to log traces and spans, define and compute evaluation metrics, score LLM outputs, compare performance across app versions, and more. As a true open-source project, its full featureset is available for use by anyone, completely free.

Contributor is looking for a community manager! If you want to know more, shoot us an email at eric@scalevp.com.

Subscribe to Contributor on Substack for email notifications!

In this episode we discuss:

  • How Opik’s popularity blew up beyond the Comet team’s expectations

  • Why CI/CD is especially important in an end-to-end platform

  • Gideon’s “severe allergy” to “fake open-source” offerings

  • Why the number of dedicated machine learning engineers is actually going down

  • Eric’s thoughts on what it means for venture capital to invest in the LLM space

Links:

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode