Deep Papers cover image

Breaking Down EvalGen: Who Validates the Validators?

Deep Papers

00:00

Evaluating Language Model Outputs Using the E-Valgen Framework

Exploring the EvalGen workflow for evaluating language model outputs, including criteria inference, manual selection, and grading options. Emphasizing the importance of tailoring evaluations to individual needs, refining assertions iteratively, and involving human feedback in the assessment loop. Discussing challenges in defining assertions, customizing metrics, and using LOM judges for production environment assessment.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app