Learning superhuman reward models or safety classifiers from weak supervision would be a significant advancement for super alignment. It is feasible to elicit key capabilities from a strong model using a weak supervisor, leading to consistently outperforming the weak model. Generalization appears to be a promising approach to alignment, although directly fine-tuning a big model to imitate a small model is suboptimal. Nudging the generalization towards outputting what it internally knows drastically improves weak to strong generalization performance. By fine-tuning GPT-4 using a GPT-2 level supervisor, performance close to GPT-3.5 can be attained on NLP tasks.
OpenAI's Superalignment team, launched this summer, has just published their first paper about weak-to-strong generalizations, and how they can analogize using weaker models to train more advanced models to simulate humans trying to control superhuman AI. Before that on the Brief, Intel's latest in the AI chip race.
Today's Sponsors:
Listen to the chart-topping podcast 'web3 with a16z crypto' wherever you get your podcasts or here: https://link.chtbl.com/xz5kFVEK?sid=AIBreakdown
ABOUT THE AI BREAKDOWN
The AI Breakdown helps you understand the most important news and discussions in AI.
Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe
Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown
Join the community: bit.ly/aibreakdown
Learn more: http://breakdown.network/