Get the app
Kelly Hong
Researcher at Chroma, working on vector databases and research around retrieval. Focuses on improving the evaluation of AI systems and retrieval methods.
Best podcasts with Kelly Hong
Ranked by the Snipd community
146 snips
Apr 23, 2025
• 54min
Generative Benchmarking with Kelly Hong - #728
chevron_right
Kelly Hong, a researcher at Chroma, delves into generative benchmarking, a vital approach for evaluating retrieval systems with synthetic data. She critiques traditional benchmarks for failing to mimic real-world queries, stressing the importance of aligning LLM judges with human preferences. Kelly explains a two-step process: filtering relevant documents and generating user-like queries to enhance AI performance. The discussion also covers the nuances of chunking strategies and the differences between benchmark and real-world queries, advocating for a more systematic AI evaluation.
The AI-powered Podcast Player
Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
Get the app