Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind
Dwarkesh Podcast
Navigating Long Context Evaluations in Language Models
This chapter explores the complexities of long context evaluations in language models, emphasizing the need to exclude certain training data for accurate assessments. The discussion also addresses evaluation methods from the Gemini 1.5 paper, including the potential of unsupervised loss functions, while highlighting issues like reward function hacking and the subjective nature of human preferences in response evaluation.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
