22 - Shard Theory with Quintin Pope
AXRP - the AI X-risk Research Podcast
The Differences Between Self-Supervised Learning and RL in Deep Learning
Almost seems like you think that if you're learning from a bunch of data basically any learning rule you can do is good enough to learn the entire data set. It's not like they're exactly equivalent there are some differences in how they tend to generalize or of course how quickly they tend to learn the data so like atom versus SGD is one example, Atom is much better at learning data but it doesn't generalize as well as SGD normally does. You can also go up an order and be like do second order corrections to SGD such as Hessian-based or yeah Newton method optimizer sort of stuff at a Hessian.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
