LessWrong (Curated & Popular) cover image

"Why I think strong general AI is coming soon" by Porby

LessWrong (Curated & Popular)

00:00

The Importance of a Simple Feed-Forward Network

Transformers can even be thought of as a special case of graph neural networks. It's quite possible that some of the things that make a transformer a transformer aren't actually critical to its performance, and a simpler model could do just as well. The stronger your optimizer, or the more compute you have, the less you need to worry about providing a fine-tuned structure.

Play episode from 12:49
Transcript

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app